Source-linked AI summary
LLMCrater: Lifecycle-Aware FAIR Metadata Generation using Large Language Models
Dani Termaat, Nafiseh Soveizi, Zhiming Zhao, Marios Avgeris
TL;DR
FAIR metadata for heterogeneous research assets is difficult to create and maintain, while publication-time workflows can miss lifecycle context. LLMCrater combines large language models with stage-specific RO-Crate profiles to progressively generate metadata across the research lifecycle. In two use cases, it generated valid RO-Crates that passed validation and were packaged as publication-ready research objects.
Problem
Publication-time FAIR metadata workflows can miss evolving contextual and provenance information for heterogeneous research assets.
Method
LLMCrater combines large language models with stage-specific RO-Crate profiles to extract, generate, validate, and package metadata across four lifecycle phases.
Results
In two heterogeneous use cases, the generated RO-Crates passed validation and were packaged as publication-ready research objects.
Takeaways & Limitations
Progressive lifecycle capture preserves contextual information and produces machine-actionable research objects ready for FAIR publication and reuse.
Abstract
from arXiv · showhide
FAIR (Findable, Accessible, Interoperable, and Reusable) metadata is essential for the discovery, interoperability, and reuse of scientific research assets. However, creating and maintaining FAIR metadata remains largely manual, making the process time-consuming for heterogeneous research artifacts generated throughout the research lifecycle. Existing approaches primarily generate metadata at publication time, missing opportunities to capture contextual information as it becomes available. To address this limitation, we present \emph{LLMCrater}, a lifecycle-aware metadata generation framework that combines Large Language Models (LLMs) with stage-specific RO-Crate metadata profiles. The framework progressively enriches metadata across four research lifecycle stages (Design, Development, Deployment, and Execution \& Provenance) while remaining compatible with RO-Crate~1.1 and EOSC metadata recommendations. It automatically extracts metadata from heterogeneous artifacts, generates and validates machine-actionable RO-Crates, and supports publication to FAIR repositories and PID services (e.g., Zenodo). We demonstrate the approach using two representative use cases: a 5G experimentation environment within SLICES-RI and an experiment on GreenDIGIT's EcoJupyter platform. Results show that LLMCrater progressively enriches metadata throughout the research lifecycle and generates valid RO-Crates conforming to the RO-Crate~1.1 specification.
I. INTRODUCTION
Research infrastructures produce heterogeneous digital assets whose FAIR metadata evolves across the research lifecycle. Publication-time workflows and labor-intensive compliance can miss contextual information needed for earlier sharing and collaboration.
- Research infrastructures continuously produce diverse assets, including datasets, software, workflows, notebooks, configurations, and experimental outputs.
- Metadata becomes available at different lifecycle stages, from project objectives and software to infrastructure details, provenance, and execution records.
- Publication-only metadata generation can miss contextual and provenance information that supports progressive FAIRification, early sharing, and collaboration.
- RO-Crate and EOSC recommendations provide a foundation for interoperable, machine-actionable research objects, but compliant metadata creation remains labor-intensive.
- LLMCrater combines large language models with stage-specific RO-Crate profiles to progressively generate metadata across four lifecycle phases.
II. LIFECYCLE-AWARE METADATA GENERATION
LLMCrater represents research objects through progressively enriched RO-Crates across four lifecycle phases. Each stage adds newly available metadata while preserving compatibility with RO-Crate 1.1 and EOSC recommendations.
- Four stage-specific profiles define mandatory and optional metadata for Design, Development, Deployment, and Execution & Provenance.
- At each lifecycle stage, LLMCrater creates a new RO-Crate that extends its predecessor with newly available metadata rather than replacing it.
- The resulting sequence represents a research project through progressively richer RO-Crates, each capturing a specific lifecycle state.
- All lifecycle profiles remain compatible with RO-Crate 1.1 and EOSC metadata recommendations.
- Incremental capture preserves contextual information and produces machine-actionable research objects ready for FAIR publication and reuse.
III. LLMCRATER ARCHITECTURE
LLMCrater uses lifecycle-aware profiles and a retrieval-augmented generation workflow to create metadata for heterogeneous research artifacts. Generated JSON-LD is validated and packaged as RO-Crates, with optional Zenodo publication.
- The architecture generates and enriches FAIR metadata for datasets, software repositories, notebooks, workflows, infrastructure configurations, and documentation.
- The current lifecycle phase selects the corresponding stage-specific RO-Crate profile that guides metadata generation.
- LLMCrater uses retrieval-augmented generation to combine lifecycle-profile and RO-Crate 1.1 guidance with research artifacts for metadata generation.
- The generated JSON-LD metadata is validated with the RO-Crate Python library before packaging with research assets as an RO-Crate.
- The architecture can optionally publish resulting research objects to Zenodo, where they receive persistent DOIs.
IV. EXPERIMENTAL EVALUATION
The proof of concept evaluates LLMCrater on two heterogeneous research-infrastructure use cases. In both cases, it extracted, generated, validated, and packaged metadata as publication-ready RO-Crates.
- The evaluation covers SLICES-RI’s 5G experimentation environment and GreenDIGIT’s federated EcoJupyter platform.
- The use cases include notebooks, datasets, software repositories, workflow descriptions, infrastructure configurations, and documentation.
- For each use case, LLMCrater extracted metadata, selected a lifecycle profile, generated JSON-LD, validated the RO-Crate, and packaged the research object.
- In both case studies, the generated RO-Crates passed validation and were packaged as publication-ready research objects.