Source-linked AI summary
Exploring Agentic Approaches for Data Issue Detection and Repair in AI-Assisted Visualization
Parimal Kashireddy, Anna Fariha, Mahmood Jasim
TL;DR
AI-assisted visualization can produce plausible but misleading results when underlying data issues go undetected or are incorrectly repaired. This preliminary study evaluates commercial LLMs on injected issues in 911 emergency-call data across single-agent and multi-agent workflows. Models handled some single-field issues but remained unreliable on cross-field, externally grounded, and structural issues, motivating explicit assumptions, selective human review, and evidence-based repair.
Problem
Subtle data issues can produce plausible but misleading visualizations, creating detection, diagnosis, and repair challenges for non-experts.
Method
The study evaluates four commercial LLMs on five injected issues in a curated 911 dataset using three prompting stages and single-agent or five-agent workflows.
Results
LLMs identified some single-field issues but remained unreliable for cross-field, externally grounded, and structural issues; multi-agent decomposition improved process traceability without consistently improving repair.
Takeaways & Limitations
Future systems should expose data assumptions, escalate ambiguous or potentially destructive repairs selectively to users, and require evidence for semantic correctness.
Abstract
from arXiv · showhide
AI is increasingly lowering the barrier to data analysis and creating visualization scripts. However, a key obstacle in AI-assisted visualization is that certain data issues can lead to visualizations that are plausible, but misrepresent the underlying data. These \textit{visualization defects} are elusive and difficult to fix, particularly for non-experts who may not know what data issues cause them or how to guide AI systems to resolve them. We present findings of a preliminary empirical investigation of how commercial LLMs identify and repair defect-inducing data issues. Using a curated subset of the 911 emergency-call dataset with five injected data issues, we evaluated GPT-5, GPT-4o, GPT-4, and Claude Sonnet 4.6 under a three-stage prompting protocol, including zero-shot, guided issue-identification, and guided issue-repair. We executed this protocol under two conditions: single-agent and a multi-agent orchestration that separates data issue detection, review, repair planning, data repair, and repair quality assurance. We observed that across both conditions, LLMs identified and repaired single-field issues (e.g., missing values) but struggled to identify and repair temporal, geographic, and semantic issues. Based on these observations, we discuss design implications for agentic visualization systems, including explicit representation of data assumptions, selective human intervention for ambiguous decisions, and evidence-based repair.
1 INTRODUCTION
AI-assisted visualization lowers barriers to analysis but remains vulnerable to subtle data issues that produce plausible, misleading visualizations. This preliminary study investigates how commercial LLMs detect and repair such issues and motivates explicit assumptions, selective human intervention, and evidence-based repair.
- Motivation: Data-issue-induced visualization defects arise when visualization pipelines make incorrect assumptions about source-data semantics, structure, or quality.Examples include missing values, inconsistent labels, and malformed dates that can misrepresent or exclude observations.
- Motivation: AI-assisted visualization systems create three challenges: detecting defects, diagnosing their data-issue causes, and resolving those issues.Existing linters and validation tools address only some explicitly encoded design or data constraints.
- Study objective: The study evaluates how GPT-5, GPT-4o, GPT-4, and Claude Sonnet 4.6 identify and repair five injected data issues in 911 emergency-call data.Models were tested with zero-shot, guided issue-identification, and guided issue-repair prompts.
- Observed challenge: LLMs reliably identified single-field issues such as missing values but struggled with cross-field, structural, and domain-constrained issues.They also sometimes produced semantically incorrect or destructive repairs that could alter communicated quantities despite making scripts executable.
- Design implications: The paper argues that future systems should expose data assumptions, support selective human intervention for ambiguous decisions, and use evidence-based repair.Successful visualization-script execution alone does not guarantee a defect-free visualization.
2 STUDY DESIGN
The study uses controlled data issues in a curated 911 emergency-call subset to compare single-agent and multi-agent LLM workflows. Its orchestration separates detection, review, planning, fixing, and quality assurance while recording evidence, guidance, and modifications.
- Dataset: The study uses 100 records and 9 fields from a randomly selected 911 emergency-call dataset subset with five manually injected data issues.The fields include timestamps, coordinates, ZIP codes, township, address, descriptions, emergency type, and an indicator.
- Conditions: The experiments compare single-agent and multi-agent conditions using GPT-5, GPT-4o, GPT-4, and Claude Sonnet 4.6.In the single-agent condition, each model performs the complete task independently.
- Multi-agent orchestration: The multi-agent orchestration uses five specialized agents: detector, detection reviewer, repair planner, data fixer, and quality assurance reviewer.The detector examines within- and across-field relationships and records suspected issues with supporting evidence.
- Multi-agent orchestration: The detection reviewer evaluates whether reported issues are supported, distinguishes uncertainty from corruption, and identifies missing checks or unavailable knowledge.This intermediary stage is intended to reduce conversion of incorrect detections into repair actions.
- Repair and verification: Repair plans specify affected records, observed values, proposed transformations, and operation types before being passed to the data fixer.The fixer modifies a copy, preserves the original CSV, limits changes to planned fields, and produces a change log for QA.
- Repair and verification: Quality assurance compares pre- and post-repair structure, planned modifications, untouched cells, missing values, and expected field types and categories.The task protocol progressively adds field or relationship guidance across zero-shot, guided identification, and guided repair stages.
3 FINDINGS FROM THE OBSERVATIONS
Across the exploratory observations, models handled some single-field issues but remained unreliable on issues requiring cross-field reasoning, structural reconstruction, or semantic constraints. Multi-agent decomposition exposed failures and sometimes reduced prompting, but did not consistently improve correct end-to-end repair.
- Single-field issues: The experiments report descriptive observed behaviors rather than model-accuracy benchmarks, defining success as correct issue identification and repair without new visualization defects.The missing-value case was detected across both conditions, but intended repair success varied by model and condition.
- Multi-agent condition: Multi-agent decomposition did not resolve difficulties recognizing and validating relationships among semantically related fields.It reduced iterative prompting for some guided tasks but still required specific guidance or failed on ZIP–location and coordinate–address contradictions.
- Guidance: Issue-specific guidance improved some intermediate detections and repairs but did not reliably produce correct repairs across temporal, ZIP-code, and delimiter cases.GPT-4 repaired a temporal issue after directed comparison, while GPT-4o repaired a corrupted ZIP code in the multi-agent condition; delimiter repair remained unsuccessful.
- Structural issues: No model successfully repaired the delimiter error in either condition, even with issue-specific guidance.The inserted comma shifted later values across columns, requiring reconstruction of the original row structure.
- Repair validity: Successful execution of a modification was insufficient evidence that the repaired data preserved intended semantics or structure.Some repairs altered data without resolving the intended issue or introduced new problems.
4 DISCUSSION
The discussion argues that reliable data repair requires explicit assumptions, selective human review, and evidence beyond agent confidence or successful execution. Multi-agent decomposition improves traceability but does not consistently ensure correct repairs.
- Agent Specialization Should not be Treated as a Correctness Mechanism: Multi-agent decomposition made failure stages more visible but did not consistently improve end-to-end issue resolution.The authors caution that stacking multiple LLMs may not provide sufficient evidence for repair.
- Agent Specialization Should not be Treated as a Correctness Mechanism: Repair actions should depend on supporting evidence rather than agent confidence or agreement alone.The discussion recommends heterogeneous checking and stronger evidence before modifying data.
- Make Latent Data Assumptions Explicit and Editable: Field relationships and expected constraints should be explicit, editable assumptions that users can validate.Explicitly providing relationships such as shared event times enabled checks that models otherwise missed.
- Treat Human Intervention as Evidence-Based Exception Handling: Systems should escalate ambiguous or potentially destructive repairs while allowing well-supported repairs to proceed automatically.They should present relevant evidence, alternatives, and expected changes for user confirmation.
- Successful Execution Should Not Be Treated as Evidence of a Correct Repair: Repair verification must distinguish execution validity, structural validity, and evidence that modified values preserve intended meaning.A quality-assurance stage can detect some failures, but execution success alone does not establish semantic correctness.
5 CONCLUSIONS
The preliminary exploration finds that LLMs handle some single-field data issues but remain unreliable on cross-field, externally grounded, and structural issues. It motivates evidence-based repair with explicit assumptions and selective user involvement.
- 5 CONCLUSIONS: LLMs identified some single-field data issues but remained unreliable on cross-field, externally grounded, and structural issues.The conclusion summarizes observations across both single-agent and multi-agent conditions.
- 5 CONCLUSIONS: Correct repair depended on detecting an issue and obtaining sufficient evidence to justify the intended modification.The authors connect repair success to both issue identification and evidential support.
- 5 CONCLUSIONS: The findings motivate repair mechanisms that make data assumptions explicit and involve users selectively when repairs are ambiguous or potentially destructive.The proposed direction combines transparent assumptions with targeted human intervention.