Source-linked AI summary
Can LLMs Clean Up Your Mess? A Survey of Application-Ready Data Preparation with LLMs
Wei Zhou, Jun Zhou, Haoyu Wang, Zhenghao Li, Qikang He, Shaokun Han, Guoliang Li, Xuanhe Zhou, Yeye He, Chunwei Liu, Zirui Tang, Bin Wang, Shen Tang, Kai Zuo, Yuyu Luo, Zhenzhe Zheng, Conghui He, Jingren Zhou, Fan Wu
TL;DR
Data preparation is essential but traditional pipelines require substantial manual effort, specialized expertise, and labeled data while generalizing poorly across domains and modalities. This survey reviews hundreds of works through a task-centric taxonomy of cleaning, integration, and enrichment, synthesizing how LLMs reshape preparation workflows. It identifies a shift toward cost-efficient hybrids and flexible prompting, alongside unresolved challenges in scalable execution and reliable deployment.
Problem
Traditional data-preparation methods require manual rules, specialized expertise, task-specific training, and labeled data, limiting scalability and generalization.
Method
The survey systematically reviews LLM-enhanced data preparation across data cleaning, integration, and enrichment, organizing methods by task and technical characteristics.
Results
The reviewed literature shows movement toward cost-efficient hybrid methods and reduced emphasis on task-specific fine-tuning, while fully studied agentic implementations remain limited.
Takeaways & Limitations
Prompting, retrieval augmentation, structured serialization, executable programs, and smaller models are emerging as ways to improve flexibility, scalability, and maintenance.
Takeaways & Limitations
Traditional preparation remains constrained by substantial manual effort and dependence on specialized expertise.
Abstract
from arXiv · showhide
Data preparation aims to denoise raw datasets, uncover cross-dataset relationships, and extract valuable insights from them, which is essential for a wide range of data-centric applications. Driven by (i) rising demands for application-ready data (e.g., for analytics, visualization, decision-making), (ii) increasingly powerful LLM techniques, and (iii) the emergence of infrastructures that facilitate flexible agent construction (e.g., using Databricks Unity Catalog), LLM-enhanced methods are rapidly becoming a transformative and potentially dominant paradigm for data preparation. By investigating hundreds of recent literature works, this paper presents a systematic review of this evolving landscape, focusing on the use of LLM techniques to prepare data for diverse downstream tasks. First, we characterize the fundamental paradigm shift, from rule-based, model-specific pipelines to prompt-driven, context-aware, and agentic preparation workflows. Next, we introduce a task-centric taxonomy that organizes the field into three major tasks: data cleaning (e.g., standardization, error processing, imputation), data integration (e.g., entity matching, schema matching), and data enrichment (e.g., data annotation, profiling). For each task, we survey representative techniques, and highlight their respective strengths (e.g., improved generalization, semantic understanding) and limitations (e.g., the prohibitive cost of scaling LLMs, persistent hallucinations even in advanced agents, the mismatch between advanced methods and weak evaluation). Moreover, we analyze commonly used datasets and evaluation metrics (the empirical part). Finally, we discuss open research challenges and outline a forward-looking roadmap that emphasizes scalable LLM-data systems, principled designs for reliable agentic workflows, and robust evaluation protocols.
I. INTRODUCTION
Data preparation transforms raw data into high-quality inputs for downstream applications, but traditional approaches face manual effort, limited semantic awareness, poor generalization, and dependence on labeled data.
- Motivation: Data preparation transforms raw datasets by denoising inputs, identifying cross-dataset relationships, and extracting meaningful insights for downstream applications.It supports applications including business intelligence analytics, machine-learning training, and data sharing.
- Motivation: Data inefficiencies arise from consistency and quality issues, integration barriers, and semantic or contextual limitations.Examples include non-standard formats, entity ambiguity, schema conflicts, missing metadata, and unlabeled data.
- Traditional limitations: Traditional preparation requires substantial manual effort and specialized expertise because it relies on fixed rules, handcrafted scripts, and manually configured constraints.Detect-then-correct workflows can also introduce new errors through incorrectly repaired values.
- Traditional limitations: Rule-based and syntactic approaches have limited semantic awareness, hindering resolution of abbreviations, synonyms, domain terminology, and user intent.These limitations affect integration matching and enrichment search.
- Traditional limitations: Traditional models generalize poorly across diverse domains and modalities, while small-model approaches depend on expensive labeled data and limited contextual integration.Fine-tuned matching models degrade on out-of-distribution entities, and annotation models struggle with underrepresented domains.
B. LLM-Enhanced DATA PREPARATION: Driving Forces And Opportunities
Rising demand for application-ready data and advances in LLMs are driving a shift from manual, rule-based preparation toward instruction-driven and agentic workflows. These methods add semantic reasoning and cross-modal adaptability while reducing reliance on task-specific engineering.
- Driving forces: Increasing demand for application-ready data and autonomous workflows is driving the shift toward LLM-enhanced data preparation.The motivating applications include personalized customer experiences and real-time analytics.
- Opportunities: LLMs enable instruction-driven and agentic automation through natural-language transformation specifications and agents that plan, execute, and orchestrate preparation workflows.Cleaning agents can identify quality issues and invoke external tools with minimal human intervention.
- Opportunities: LLM-enhanced methods incorporate semantic reasoning to resolve abbreviations, synonyms, domain terminology, semantic column groups, and dataset descriptions.This extends preparation beyond syntactic similarity, keyword search, and statistical profiling.
- Opportunities: LLM-enhanced techniques support cross-modal generalization by handling heterogeneous schemas and formats through instructions, few-shot prompting, and modality-bridging encoders.These methods reduce reliance on domain-specific feature engineering and task-specific training.
C. Contributions and Differences with Existing Surveys
The survey reviews data cleaning, integration, and enrichment through a task-centered taxonomy, contrasting LLM-enhanced methods with existing surveys and identifying methodological trends and open deployment gaps.
- Holistic task scope: The survey covers data cleaning, data integration, and data enrichment across table and text modalities.It organizes representative methods and technical characteristics within these three core tasks.
- Systematic taxonomy: Its unified taxonomy organizes LLM-enhanced methods by underlying techniques, including prompt-based and agent-based frameworks.This contrasts with prior surveys using coarser categories or narrower method-specific scopes.
- Paradigm shift: The survey traces data preparation’s evolution from rule-based systems to LLM agent frameworks, summarizing associated advantages and limitations.This provides a paradigm-level synthesis rather than only describing studies individually.
- Methodological evolution: Recent work shifts toward cost-efficient hybrid methods that generate executable programs or transfer reasoning to smaller language models.These approaches reduce execution cost and improve scalability.
- Open deployment gap: Agentic implementations remain limited, leaving reliable and robust deployment insufficiently explored.The survey also distinguishes task-specific patterns: hybrid LLM–ML for cleaning, multi-model collaboration for integration, and retrieval-augmented or hybrid prompting for enrichment.
II. DATA PREPARATION: DEFINITION AND SCOPE
The paper defines application-oriented data preparation as improving raw data for downstream use and scopes its review to cleaning, integration, and enrichment. It specifies the principal subtasks and the LLM-enhanced techniques used to address them.
- Definition and scope: Data preparation transforms raw datasets into high-quality data by denoising, identifying cross-dataset relationships, and extracting meaningful insights.The survey focuses on data used in downstream analytical and machine-learning applications rather than preparing data for LLMs themselves.
- Data Cleaning: Data cleaning produces trustworthy data through standardization, error processing, and imputation.Error processing detects erroneous values and repairs them, while imputation estimates plausible replacements for missing entries.
- LLM-enhanced techniques: LLM-enhanced preparation uses semantic reasoning, structured prompting, retrieval, external knowledge, and hybrid architectures across these subtasks.The cited techniques support error correction, missing-value completion, entity matching, annotation, and metadata generation.
- Data Integration: Data integration aligns elements across datasets, with the survey focusing on entity matching and schema matching.Entity matching links records for the same real-world entity, whereas schema matching identifies correspondences between columns or tables.
- Data Enrichment: Data enrichment augments datasets with semantic labels, descriptive metadata, or complementary-dataset relationships.The review concentrates on data annotation and data profiling as predominant enrichment operations.
III. LLM FOR DATA CLEANING
LLM-enhanced data cleaning replaces rigid, manual rules with prompt-, code-, retrieval-, and model-based workflows for standardization, error processing, and imputation. These approaches improve flexibility and semantic handling, but face scalability, hallucination, and data-quality constraints.
- Data Standardization: LLM-enhanced standardization uses structured prompts, batch-wise processing, code synthesis, or agents to transform heterogeneous values into unified formats.Prompt-based methods support rapid deployment, while reusable generated functions improve execution scalability; agentic workflows translate natural-language specifications into executable pipelines.
- Limitations: Code-based standardization offers fixed execution costs but may generate faulty functions, while agentic pipelines add maintenance overhead and latency through tool coordination.Reliability may require aggregating multiple candidate functions.
- Data Error Processing: Data error processing detects and repairs typos, anomalous values, and dependency violations through prompting, synthesized rules, fine-tuning, or lightweight models.Function-synthesis methods can derive ontological functional dependencies, while LLM labeling can create training data for efficient detectors.
- Limitations: Prompt-based cleaning can consume many tokens, iterative self-correction can compound hallucinations, and aggressive context pruning can omit relationships needed for complex imputations.Synthesizing validation rules from corrupted inputs may encode errors as valid rules, while adaptive methods require substantial high-quality training data.
- Data Imputation: LLM-enhanced imputation fills missing entries using structured prompts, retrieved context, fine-tuned models, or architecture modules that capture mixed-type relationships.Retrieval-guided systems select and rerank relevant tuples from data lakes before prompting the LLM, whereas optimized models adapt training or architecture.
IV. LLM FOR DATA INTEGRATION
LLM-enhanced data integration addresses semantic ambiguity in entity and schema matching through prompting, batching, retrieval, model optimization, collaboration, and agent orchestration. The survey emphasizes improved semantic reasoning and workflow flexibility alongside token, retrieval, training-data, and generalization constraints.
- Motivation: LLMs provide semantic understanding that helps resolve abbreviations, synonyms, domain terminology, and relationships that conventional similarity methods struggle to identify.This advantage is especially relevant when domain-specific knowledge is unavailable to traditional integration methods.
- Entity Matching: Entity matching methods use structured prompts, batch clustering, and multi-model collaboration to link records that refer to the same real-world entities.Batching groups semantically similar entity pairs, while collaborative systems combine complementary models and local-global matching strategies.
- Limitations: Prompt-based matching remains sensitive to phrasing and costly for large candidate lists, while schema matching can incur prohibitive token use during exhaustive verification.Metadata-only methods may fail on semantically opaque column names.
- Schema Matching: Schema matching methods align columns across databases using direct prompting, retrieval-enriched context, model optimization, lightweight scorers, and agent orchestration.Retrieval systems rank candidate matches using vector, graph, or query-driven context; specialized table encoders support permutation-invariant cross-table representations.
- Limitations: Retrieval-enriched matching depends on external knowledge coverage and retrieval overhead, whereas model-optimized matching shifts the bottleneck toward high-quality training data.Synthetic-data quality and transferred reasoning also constrain lightweight filter-then-rank pipelines.
V. LLM FOR DATA ENRICHMENT
LLM-enhanced data enrichment methods primarily address data annotation and profiling through prompts, reasoning, retrieval, fine-tuning, and hybrid collaboration. These approaches improve semantic understanding and metadata generation but remain constrained by cost, latency, retrieval quality, training-data demands, and information loss.
- Data Annotation: Prompt-based annotation uses explicit guidelines, contextual information, reasoning, and iterative refinement to label complex data.Examples include representative-sample annotation with clustering, stratified sampling, and human feedback.
- Data Annotation: Collaborative LLM-SLM annotation can generate multiple candidate labels, distill and filter them, and refine distributions to correct false positives on noisy data.CanDist uses task-specific prompts, RoBERTa-Base, and distribution refinement.
- Data Annotation: LLM annotation methods include prompt-based, retrieval-enriched, fine-tuned, and hybrid approaches, each trading accuracy or specificity against latency, retrieval noise, or data requirements.Fine-tuning can reduce deployment costs but requires extensive, high-quality instruction data; retrieval quality depends on external resources.
- Scope: Data enrichment augments datasets with semantic labels, descriptive metadata, or complementary datasets, with annotation and profiling as predominant operations.Annotation assigns semantic or structural labels, while profiling characterizes datasets and their relationships.
- Data Profiling: Prompt-based profiling combines instructions, constraints, examples, and reasoning to generate descriptions, summaries, or hierarchical views for complex datasets.AutoDDG targets user-oriented and search-optimized descriptions, while LEDD supports hierarchical data-lake profiling.
- Data Profiling: Profiling remains limited by lossy summary-statistic compression and information loss during iterative abstraction, which can hide fine-grained anomalies or produce generic descriptions.Hybrid profiling adds LLM reasoning and human verification for quality assurance.
A. DATA PREPARATION Datasets
The survey organizes representative data-preparation benchmarks by processing granularity, covering tuple-, column-, cell-, schema-, table-, and document-level units. These datasets span cleaning, error processing, imputation, entity matching, schema alignment, profiling, and annotation.
- Benchmark Organization: The dataset summary records category, task, modality, granularity, data volume, and evaluation metrics to support benchmark selection.The survey presents this organization as a systematic basis for comparing datasets across use cases.
- Tuple-, Column-, and Cell-Level: Tuple-level benchmarks support error detection, standardization, imputation, and entity matching across tabular and text-based datasets.Examples include Adult Income, Hospital, Beers, Flights, and Enron Emails.
- Schema-Level: Schema-level benchmarks evaluate semantic and structural alignment between attributes or schemas, including clinical and scientific cross-source matching.Examples include OMOP, Synthea, MIMIC, GDC-SM, and ChEMBL-SM.
- Object-Level: Object-level benchmarks use entire tables or documents for profiling, annotation, and broader contextual reasoning.Examples include Public BI, Adventure Works, NQ-Tables, FetaQA, AGNews, DBpedia, and CoNLL-2003.
B. DATA PREPARATION Metrics
The survey organizes evaluation metrics by the aspect they measure rather than by task alone, emphasizing correctness, robustness, ranking quality, semantic consistency, and enrichment completeness. The supplied metrics quantify coverage and semantic preservation in enrichment settings.
- Metric Organization: Evaluation metrics are grouped by correctness, robustness, ranking quality, and semantic consistency across preparation tasks.This organization reflects the multiple dimensions used in real deployments.
- Enrichment Completeness: Recall@GT measures the fraction of correctly identified elements among the top-k results, where k equals the number of true elements.It evaluates coverage in entity- or schema-matching tasks.
- Enrichment Completeness: 1 −α measures the fraction of data elements whose correct label appears among candidates, evaluating label coverage in annotation.The metric is presented as an enrichment-completeness measure.
- Enrichment Completeness: Hit Rate measures the fraction of search queries returning at least one correct result, evaluating basic retrieval success in annotation.It captures whether retrieval produces any correct candidate.
- Semantic Preservation: ROUGE assesses lexical semantic consistency by measuring n-gram overlap between generated output and reference text.It is used to evaluate whether outputs retain key terms.
A. Data Cleaning
LLM-enhanced cleaning still faces a global-context gap and coordination challenges across models with different capabilities. The survey points toward holistic context handling and hierarchical deployment to improve coverage, privacy, and cost efficiency.
- Global-Aware Cleaning: Most prompt-based cleaning methods use local row or batch contexts and therefore miss dataset-level properties such as uniqueness constraints and aggregate correlations.Retrieval augmentation expands evidence but remains centered on instance-level context.
- Hierarchical Deployment: Privacy-preserving local models and federated learning exist, but current systems lack principled coordination strategies for models with different capabilities.The proposed direction assigns routine tasks to small local models and complex reasoning to LLMs.
- Hierarchical Deployment: Hierarchical cleaning frameworks could combine local models, LLMs, table partitioning, and selective context management to reduce cost and latency.This design preserves LLM use for complex reasoning while distributing routine work to smaller models.
B. Data Integration
LLM-enhanced data preparation addresses integration settings ranging from extreme schema heterogeneity to domain-dependent enrichment decisions. The survey organizes these challenges and reviews the need for workflows that support both broad integration and interactive human guidance.
- Universal and Cross-Domain Integration: Extreme heterogeneity—including abbreviated attributes, structural mismatches, and little usable metadata—limits current cross-domain integration methods.Existing structure-aware matching and cross-dataset integration studies generally assume reasonably informative schemas.
- Interactive Human-in-the-Loop Enrichment: Ambiguous or domain-dependent enrichment decisions make fully automated data enrichment impractical in many real-world settings.
- Interactive Human-in-the-Loop Enrichment: Effective enrichment workflows require close collaboration between human experts and LLM-enhanced systems.Users need to guide decisions, verify results, and correct errors during enrichment.
- Interactive Human-in-the-Loop Enrichment: Most existing enrichment methods are one-shot automations with limited support for interactive refinement and explanation.The survey calls for frameworks that let LLMs explain reasoning and solicit user input.
- The survey uses a task-centric review to organize data cleaning, data integration, and data enrichment while synthesizing methods, datasets, metrics, limitations, and future directions.