PaperlayerView exact revision on arXiv

Source-linked AI summary

CoDA: Agentic Systems for Collaborative Data Visualization

Zichen Chen, Jiefeng Chen, Sercan Ö. Arik, Misha Sra, Tomas Pfister, Jinsung Yoon

arXiv:2510.03194v1cs.AI

TL;DR

Data visualization automation must address complex data and iterative refinement that challenge existing approaches. CoDA uses specialized agents for understanding, planning, generation, and reflection, and reports substantial benchmark gains over strong baselines. Its conclusion notes computational overhead from multi-turn agent communication.

  • Problem

    Existing visualization systems struggle with complex or multi-file data and iterative refinement, despite analysts spending substantial time on manual visualization tasks.

    Source: §1 · p. 1

  • Method

    CoDA uses specialized agents for metadata analysis, planning, code generation and debugging, and self-reflection in a collaborative workflow.

    Source: §1 · p. 2

  • Results

    24.5%, 41.5%, and 26.5% were the maximum Overall Score improvements reported across MatplotBench and Qwen Code Interpreter benchmarks over strong baselines.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The conclusion says metadata-centric preprocessing and self-reflection refinement help CoDA manage messy multi-file data and let analysts prioritize insights over manual work.

    Source: §6 · p. 11

    CoDA’s multi-turn agent communications incur computational overhead.

    Source: §6 · p. 11

Abstract

from arXiv · show

Deep research has revolutionized data analysis, yet data scientists still devote substantial time to manually crafting visualizations, highlighting the need for robust automation from natural language queries. However, current systems struggle with complex datasets containing multiple files and iterative refinement. Existing approaches, including simple single- or multi-agent systems, often oversimplify the task, focusing on initial query parsing while failing to robustly manage data complexity, code errors, or final visualization quality. In this paper, we reframe this challenge as a collaborative multi-agent problem. We introduce CoDA, a multi-agent system that employs specialized LLM agents for metadata analysis, task planning, code generation, and self-reflection. We formalize this pipeline, demonstrating how metadata-focused analysis bypasses token limits and quality-driven refinement ensures robustness. Extensive evaluations show CoDA achieves substantial gains in the overall score, outperforming competitive baselines by up to 41.5%. This work demonstrates that the future of visualization automation lies not in isolated code generation but in integrated, collaborative agentic workflows.

1. Introduction

Automating visualization matters because analysts spend substantial time on manual preparation and refinement, while existing systems struggle with complex data and iterative tasks. CoDA combines specialized agents and reports gains over competitive baselines.

Motivation

  • Analysts may spend over two-thirds of their time on low-level data preparation and visualization, motivating automation from natural-language queries and complex data.The stated challenges include large datasets, coordination across linguistic, statistical, and design expertise, and iterative feedback.Source: §1 · p. 1

Research gap

  • Existing rule-based, LLM-based, and multi-agent systems struggle with diverse data, token limits, multi-source integration, or persistent iterative refinement.The paper argues that systems centered on initial query parsing are insufficient for large or multiple files, code errors, and refinement.Source: §1 · pp. 1–2

CoDA

  • CoDA assigns specialized agents to metadata analysis, planning, code generation and debugging, and self-reflection to handle complex visualization needs.It analyzes schemas and statistics without uploading raw files to bypass context-window limits; Figure 1 qualitatively compares its outputs with MatplotAgent, VisPath, and CoML4VIS.Source: §1 · p. 2, Figure 1 · p. 2

Key contributions

  • CoDA’s Overall Score gains reached maximum improvements of 24.5%, 41.5%, and 26.5% across MatplotBench and Qwen Code Interpreter benchmarks over strong baselines.The introduction also reports significant positive impacts from self-evolution, the global TODO list, and example search, and says CoDA significantly outperforms baselines on DA-Code.Source: §1 · p. 3

2. Related Work

Prior work spans natural-language visualization methods and agentic systems, but the paper identifies gaps in handling ambiguity, multiple data sources, and persistent reflection.

Natural Language to Visualization

  • Natural-language visualization methods support exploration and reporting, but the literature identifies challenges in coherence, fidelity, ambiguity, and multi-source data.Several methods use chain-of-thought prompting to improve LLM accuracy on single-table tasks.Source: §2 · p. 3

Agentic Visualization Systems

  • Agentic visualization systems distribute problem-solving across agents, with prior work using multi-step reasoning, feedback optimization, and design patterns.The cited work describes these systems as streamlining visualization and reducing manual effort.Source: §2 · p. 3
  • The paper argues that many agentic systems focus adaptation on initial planning without persistent reflection, leaving vulnerabilities in complex scenarios.CoDA is presented as using specialized planning, building, criticism, and reflection agents.Source: §2 · p. 3

3. Method

CoDA frames visualization as collaborative problem solving: specialized agents share context and use quality-driven feedback to iteratively produce and refine visualizations.

3.2. CoDA: Collaborative Data Visualization Agents

  • The workflow moves from query and metadata understanding through planning, code generation and debugging, then image-based quality evaluation and refinement.Agents include a Query Analyzer, Data Processor, VizMapping, Search Agent, Design Explorer, Code Generator, Debug Agent, and Visual Evaluator.Source: Figure 2 · p. 4, §3.2 · pp. 5–6

3.1. The Collaborative Multi-Agent Paradigm

  • Conventional systems treat visualization as a monolithic, single-pass task, while CoDA unifies query understanding, data handling, code generation, and quality assurance in a self-reflection workflow.The paper says this approach operationalizes its principles for real-world benchmarks.Source: §3.1 · pp. 4–5
  • CoDA’s design principles include role specialization, metadata summaries instead of full data loading, image-based reflection, and interchangeable modules.Agents communicate through shared state to adapt dynamically; reflection can detect and correct visual clutter or factual inaccuracies.Source: §3.1 · pp. 4–5

3.2. CoDA: Collaborative Data Visualization Agents

  • Table 1 summarizes the inputs and outputs of agents in a system that takes a natural-language query and data files and produces a refined visualization.The framework overview is shown in Figure 2.Source: Figure 2 · p. 4, §3.2 · p. 5, Table 1 · p. 5
  • Agents pass structured messages through shared memory, and low quality scores route issues upstream until quality converges or reflection limits are reached.The modular design also supports parallelization and extension to tasks such as scientific plotting.Source: §3.2 · p. 6

4. Experiments

CoDA is evaluated on visualization benchmarks and repository-based software-engineering tasks, using measures of execution reliability, visualization quality, and overall success. Reported comparisons favor CoDA, while backbone and efficiency analyses describe performance across models and computational costs.

4. Experiments

  • The experiments compare CoDA with visualization baselines using standardized metrics for execution reliability, visualization correctness, and overall task success.Experiments use gemini-2.5-pro, with at most three refinement iterations and a quality threshold of 0.85 for halting.Source: §4 · p. 6

4.1. Benchmarks

  • The benchmarks cover matplotlib visualization tasks, code-interpreter visualization queries, and repository-based visualization work.MatplotBench has 100 queries, the Qwen visualization subset has 163 examples, and DA-Code (vis) has 78 tasks and is evaluated separately as a more difficult SWE-oriented setting.Source: §4.1 · p. 6

4.2. Baselines

  • Baselines include a single-agent code-synthesis system, a multiple-solution planner, and a workflow-centric visualization framework, compared with a shared gemini-2.5-pro backbone.The baseline setups follow their papers’ parameters, including iteration limits.Source: §4.2 · pp. 6–7

4.4. Main Results

  • 24.5% OS on MatplotBench and 7.4% OS on the Qwen Code Interpreter benchmark are CoDA’s gains over the best alternative.CoDA outperforms all baselines across metrics on these benchmarks.Source: Table 2 · p. 7, §4.4 · p. 7

4.3. Evaluation Metrics

  • The evaluation measures execution reliability, visualization quality among executable outputs, and overall task success through complementary metrics.On binary-scored benchmarks, VSR is the proportion of correct visualizations among executable cases.Source: §4.3 · p. 7

4.5. Results on DA-Code Benchmark

  • 39.0% is CoDA’s score on DA-Code, a 19.77% absolute gain over DA-Agent with gemini-2.5-pro.The evaluation covers visualizations embedded in broader codebases; the reported agent workflow addresses integration conflicts.Source: Table 3 · p. 8, §4.5 · p. 8

4.6. Performance with Different Backbone LLMs

  • CoDA with gemini-2.5-flash reaches 77.7% OS on MatplotBench, 1.8% below gemini-2.5-pro, while baselines do not exceed 65.2% OS.claude-4-sonnet reaches 75.2% OS, a 4.3% drop from gemini-2.5-pro; CoDA’s EPR and VSR remain 98.0–99.0% and 76.7–80.3% across backbones.Source: §4.6 · pp. 8–9

4.7. Efficiency Analysis

  • CoDA uses 17.6% fewer total tokens (50,219 vs. 60,969) and 3.9% fewer LLM calls than MatplotAgent, while achieving 79.5% vs. 51.0% OS.Simpler methods trend toward lower costs but diminished visualization quality; CoML4VIS uses 1.0 call and 6,138 tokens for 62.6% OS.Source: Table 5 · p. 9, §4.7 · p. 9

5. Ablation Study

The ablations examine iterative self-reflection, structured planning through the global TODO list, and code-example retrieval on MatplotBench. They report accuracy, overall-score, and efficiency trade-offs across these components.

5. Ablation Study

  • The controlled MatplotBench ablations use gemini-2.5-pro and separately vary refinement loops, the global TODO list, and code-example retrieval while retaining the core multi-agent pipeline.The study evaluates component effects on OS and frames them as design trade-offs, including accuracy and efficiency.Source: §5 · pp. 9–10

5.1. Impact of Self-Evolution

  • OS rises from 75.6% at 1 iteration to 79.5% at 3 (CoDA’s default), then reaches 80.1% at 5, with only a 0.6% gain from 3 to 5.EPR increases by 8.0% from 1 to 3 iterations and then stabilizes near 100%; latency rises beyond 3 iterations without proportional accuracy benefits.Source: §5.1 · p. 10, Figure 3 · p. 10

5.2. Role of Global TODO List

  • Removing the global TODO list lowers OS to 75.1% (-4.4% absolute) and EPR by 5.0%, while VSR remains stable.The authors connect the drop to fragmented intent extraction and incomplete workflows, including missed statistical insights.Source: §5.2 · p. 10, Figure 3 · p. 10

5.3. Effectiveness of Example Search Agent

  • Disabling the Search Agent lowers OS to 76.0% (-3.5%) and EPR by 9.0%, primarily through syntax errors in specialized visualizations.Retrieved ranked snippets ground coding knowledge in specific problems without post-training.Source: §5.3 · pp. 10–11, Figure 3 · p. 10

6. Conclusion

CoDA is a multi-agent framework for natural-language visualization queries, combining specialized stages for task and data understanding, planning, code generation, and self-reflection. The conclusion reports gains of up to 41.5% over named baselines and identifies multi-turn agent communication as a computational limitation.

6. Conclusion

  • CoDA decomposes natural-language visualization queries into specialized stages for data understanding, planning, code generation, and self-reflection.The conclusion says metadata-centric preprocessing and refinement address input token limits and messy multi-file data.Source: §6 · p. 11
  • CoDA reports up to 41.5% accuracy gains over MatplotAgent, VisPath, and CoML4VIS on MatplotBench and Qwen benchmarks.The conclusion attributes the approach to metadata-centric preprocessing and self-reflection refinement for messy multi-file data.Source: §6 · p. 11
  • Multi-turn agent communications impose computational overhead, which the conclusion identifies as a key limitation.The authors name agent distillation and adaptation to multimodal inputs as future directions.Source: §6 · p. 11

A. CoDA Workflow and Implementation Details

The workflow maps a natural-language query and data files through specialized agents to a refined visualization. It combines query analysis, data processing, chart and design planning, code generation, debugging, and iterative evaluation.

A. CoDA Workflow and Implementation Details

  • The workflow initializes query, processing, visualization-mapping, search, design, code, debugging, and evaluation agents.The query agent derives visualization intents and a task list from the query and metadata.Source: A. CoDA Workflow and Implementation… · p. 15
  • The processor extracts data information, insights, and processing hints; the visualization mapper uses these with the query and task list to propose charts, styles, transformations, and goals.Source: A. CoDA Workflow and Implementation… · p. 15
  • The search agent retrieves optional code examples, while the design agent proposes design specifics, quality metrics, and success indicators for code generation.The code agent generates executable, documented code using the design, processed data, and retrieved examples.Source: A. CoDA Workflow and Implementation… · p. 15
  • The debugger produces a plot and fixed code; the evaluator assesses accuracy, readability, layout, and aesthetics, with the loop ending at success criteria or when convergence or budget is reached.The workflow checks whether the overall score meets a threshold and the design’s success indicators are satisfied.Source: A. CoDA Workflow and Implementation… · p. 15

B. Additional Visualization Examples

The appendix presents additional CoDA visualization examples from DA-Code and MatplotBench, pairing each query with a ground-truth visualization and CoDA’s output. The examples illustrate complex patterns, ambiguous queries, and multi-file inputs.

B. Additional Visualization Examples

B.1.1. Example 1

Example 1 asks CoDA to produce a line chart of total NBA points by year from 1950 to 2018. It reproduces the reference visualization despite the absence of a direct total-points-per-season field.

B.1.1. Example 1

  • Because no dataset directly contains total league points per year, manually aggregate all scoring fields by season to compute Total Points Scored for 1950–2018; lockout handling is human judgment.Source: §B.1.1 · p. 16

Our Generated Code

  • The workflow reads plotting specifications from plot.yaml, then renders the annual totals as a styled line chart and saves result.png.The chart uses year on the x-axis and total points on the y-axis.Source: §B.1.1 · p. 16, Our Generated Code · p. 18

B.1.1. Example 1

  • 100/100: CoDA reproduces the reference visualization after aggregating season-level points, despite no direct “total score per season” field.The comparison is shown in Figure 4.Source: §B.1.1 · pp. 16–17, Figure 4 · p. 17

Our Generated Code

  • Generated code cleans player-season records, filters years 1950–2018, and sums points by year; it also fills missing PTS values with zero.For traded players, it keeps the first record per player and year, described as the summary row.Source: Our Generated Code · p. 18

B.1.2. Example 2

Example 2 tests CoDA on a Steam visualization task involving multiple heterogeneous data sources. The system integrates the fragmented files and reproduces the reference visualization with a score of 100/100.

Our Generated Code

  • Generated code filters games to positive playtime and ratings, standardizes both variables, and calculates their Pearson correlation for a scatter plot.The task specifies the correlation between standardized average playtime and standardized positive ratings.Source: §B.1.2 · p. 19, Our Generated Code · p. 21

B.1.2. Example 2

  • The Steam task requires manually selecting relevant tables, integrating attributes around core game identifiers, and constructing metrics aligned with the visualization goal.The data do not document dependencies across tables or provide a unified schema.Source: §B.1.2 · p. 19

Our Generated Code

B.1.2. Example 2

  • 100/100: CoDA reproduces the Steam reference visualization despite fragmented files and the absence of a unified schema.Figure 5 compares the generated visualization with the reference output.Source: Figure 5 · p. 20, §B.1.2 · p. 20

B.2.1. Example 1

This section presents an example of CoDA’s prompt design and evaluation for a hierarchical sunburst task. It covers chart mapping, code generation and debugging, design requirements, feedback-driven refinement, and semantic and quality assessment.

B.2.1. Example 1

  • 100/100: CoDA’s sunburst matches the ground truth, organizing countries under continents with population-sized segments and life-expectancy colors.The red-to-blue color scale uses the population-weighted average lifespan as its central value.Source: §B.2.1 · pp. 22–23, Figure 6 · p. 23

C. Judging Prompts and Model Setup

  • The evaluation uses gemini-2.5-pro as an LLM judge, adapting MatplotBench and Qwen Code Interpreter prompts to score code and visualization quality.MatplotBench averages the two scores; Qwen uses a combined binary 100/0 judgment, and non-executable code scores 0.Source: C. Judging Prompts and Model Setup · p. 23

D. Prompts Used in CoDA

  • CoDA’s prompts assign specialized agent roles, standardize structured outputs, and use prior results and a global TODO list to maintain workflow coherence.Prompted roles cover query analysis, data processing, visualization mapping, search, design, code generation, debugging, and visual evaluation.Source: D. Prompts Used in CoDA · pp. 24–25

B.2.1. Example 1

  • The query-mapping prompt requests a JSON chart configuration, applies query and data rules to mappings and aggregations, and requires exact column names.It prefers a requested chart type unless unsuitable and maps ordered fields to x, numeric measures to y, and categories to color.Source: D. Prompts Used in CoDA · pp. 26–27
  • The code-generation prompt requires complete, self-contained matplotlib examples that follow official documentation patterns and avoid invented APIs.It specifies matplotlib.pyplot, sample data and styling, and plt.show() in the output.Source: D. Prompts Used in CoDA · p. 27
  • The design prompt requests specifications for audience, accessibility, visual hierarchy, color, layout, typography, interaction, and technical constraints.It frames the analysis around the original query, interpreted intent, visualization type, and design TODOs.Source: D. Prompts Used in CoDA · p. 28
  • The design self-reflection prompt uses visual feedback, quality issues, and a target threshold to identify changes when the current score is below threshold.It directs the designer to revise the color scheme when feedback identifies color issues.Source: D. Prompts Used in CoDA · p. 29

D. Prompts Used in CoDA

  • The refinement prompt responds to layout, typography, or aesthetic feedback while preserving design coherence and returning updated specifications as JSON.It asks the agent to address the specific feedback and choose an approach that maintains design coherence.Source: D. Prompts Used in CoDA · p. 29
  • The code-generation prompt requests a concise plan informed by processed-data shape, columns, and quality score, then specifies architecture, plotting, validation, error handling, and quality requirements.Its requested JSON covers functions and class structure, Matplotlib choices, data preparation and validation, error handling, and code style and performance priorities.Source: D. Prompts Used in CoDA · p. 29
  • A debugging prompt directs a specialist to search the internet for a complete fix and return the error type, root cause, affected elements, and corrected Matplotlib code in JSON.The requested assessment also includes missing requirements and error location.Source: D. Prompts Used in CoDA · p. 30
  • The semantic-validation prompt checks query alignment, mathematical accuracy, visual-element compliance, layout, and professional standards, with low scores required when a plot misses the query regardless of aesthetics.It also calls for detailed JSON ratings of semantic accuracy and overall quality, requirement coverage, accessibility, and a final approve/revise/reject recommendation.Source: D. Prompts Used in CoDA · pp. 30–31
Loading 2510.03194v1…