PaperlayerView exact revision on arXiv

Source-linked AI summary

VisCoder: Fine-Tuning LLMs for Executable Python Visualization Code Generation

Yuansheng Ni, Ping Nie, Kai Zou, Xiang Yue, Wenhu Chen

arXiv:2506.03930v2cs.SEcs.AIcs.CL

TL;DR

Existing visualization-code training data lacks sufficient visual grounding, runtime validation, and supervision for recovering from execution failures. VisCode-200K combines over 200K supervised examples of runtime-validated Python plotting code paired with rendered plots and Code-Feedback revision dialogues, then fine-tunes Qwen2.5-Coder-Instruct and evaluates self-debugging. VisCoder-3B and 7B improve average execution pass rates over Qwen2.5-Coder by 19.6 and 14.5 points, respectively, while VisCoder-7B surpasses GPT-4o-mini on Seaborn and Plotly under the default setting.

  • Problem

    Existing visualization-code training data lacks sufficient visual grounding, runtime validation, and supervision for recovering from execution failures.

    Source: §1 · p. 1

  • Method

    VisCode-200K combines over 200K supervised examples of runtime-validated Python plotting code paired with rendered plots and Code-Feedback revision dialogues, then fine-tunes Qwen2.5-Coder-Instruct and evaluates self-debugging.

    Source: §1 · pp. 1–2

  • Results

    VisCoder-3B and 7B improve average execution pass rates over Qwen2.5-Coder by 19.6 and 14.5 points, respectively, while VisCoder-7B surpasses GPT-4o-mini on Seaborn and Plotly under the default setting.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Combining domain-specific executable examples with feedback-driven correction supports more reliable visualization code generation and iterative repair across plotting libraries.

    Source: §6 · p. 8

    The study covers only Python, finds comparatively weaker Plotly performance, and relies on an automatic judge without independent bias or reliability analysis.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Large language models (LLMs) often struggle with visualization tasks like plotting diagrams, charts, where success depends on both code correctness and visual semantics. Existing instruction-tuning datasets lack execution-grounded supervision and offer limited support for iterative code correction, resulting in fragile and unreliable plot generation. We present VisCode-200K, a large-scale instruction tuning dataset for Python-based visualization and self-correction. It contains over 200K examples from two sources: (1) validated plotting code from open-source repositories, paired with natural language instructions and rendered plots; and (2) 45K multi-turn correction dialogues from Code-Feedback, enabling models to revise faulty code using runtime feedback. We fine-tune Qwen2.5-Coder-Instruct on VisCode-200K to create VisCoder, and evaluate it on PandasPlotBench. VisCoder significantly outperforms strong open-source baselines and approaches the performance of proprietary models like GPT-4o-mini. We further adopt a self-debug evaluation protocol to assess iterative repair, demonstrating the benefits of feedback-driven learning for executable, visually accurate code generation.

1 Introduction

VisCoder addresses the difficulty of generating executable and semantically meaningful visualization code by combining validated visual examples with feedback-driven correction data. Fine-tuning Qwen2.5-Coder-Instruct on this corpus yields strong benchmark and self-debugging results.

Motivation

  • Visualization generation requires grounding natural-language instructions, input data structures, and rendered visual outputs, while execution success alone does not ensure semantic correctness.Existing instruction-tuning datasets generally lack visual grounding, runtime validation, and supervision for recovery from failure.Source: §1 · p. 1

Contribution

  • VisCode-200K combines executable visualization code paired with rendered plots and multi-turn correction dialogues for Python visualization and code repair.The dataset supports both initial generation and iterative refinement through runtime feedback.Source: §1 · pp. 1–2

Method

  • VisCoder fine-tunes Qwen2.5-Coder-Instruct at 3B and 7B scales and evaluates executable generation with PandasPlotBench and iterative self-debugging.The self-debug mode gives models multiple opportunities to revise failed outputs using execution traces.Source: §1 · p. 2

Results

  • 19.6 and 14.5 points: VisCoder-3B and 7B improve average execution pass rate over Qwen2.5-Coder.VisCoder-7B surpasses GPT-4o-mini on Seaborn and Plotly in the default setting and approaches GPT-4o after self-debugging.Source: §1 · p. 2

2 Related Work

Prior visualization-code research evaluates generation, cross-modal understanding, and feedback-based refinement, but training supervision grounded in execution outcomes remains limited. VisCode-200K targets this gap by combining validated visualization examples with correction traces.

LLMs for Visualization Code Generation

  • Prior work evaluates executable visualization generation across plotting libraries and explores reverse code generation from rendered charts, but persistent challenges remain in semantic grounding and API correctness.These efforts include PandasPlotBench, MatPlotAgent, VisEval, Plot2Code, and ChartMimic.Source: §2 · p. 2

Execution Feedback and Code Correction

  • Execution feedback and multi-turn correction have been studied for general code generation, while visualization systems have explored refinement from rendered outputs.However, execution-grounded revision traces have rarely been used to train visualization code generators.Source: §2 · p. 2

Position of this work

  • VisCode-200K addresses the training gap by validating plotting code and integrating multi-turn correction data during instruction construction.Its pipeline links code filtering, runtime checks, rendered plots, structured instructions, and iterative refinement supervision.Source: §2 · p. 2, Figure 1 · p. 3

3 VisCode-200K: A Python Visualization Instruction Tuning Dataset

VisCode-200K is built as a Python visualization instruction-tuning corpus combining validated plotting examples with multi-turn execution-feedback dialogues. Its pipeline standardizes data and plot descriptions, validates executable outputs, and supplies supervision for iterative correction.

3 VisCode-200K: A Python Visualization Instruction Tuning Dataset

  • VisCode-200K combines executable visualization examples from public Python repositories with multi-turn Code-Feedback dialogues to support generation and iterative refinement.The corpus is designed for diverse plotting libraries and realistic correction behaviors.Source: §3 · p. 3, §3.2 · p. 4

3.1 Code Extraction from Public Repositories

  • The extraction pipeline filters visualization libraries, extracts code blocks, validates execution, and generates structured instructions.Figure 1 summarizes these four construction stages.Source: §3 · p. 3, §3.1 · p. 3
  • The public-code sources are stack-edu and CoSyn-400K, covering common libraries and complementary data representations.Stack-edu embeds mock data in code, whereas CoSyn stores code and data separately and therefore requires reconstruction.Source: §3.1 · pp. 3–4
  • Approximately 1.7M stack-edu samples are filtered to about 1M candidate blocks and then curated to 300K visualization blocks, while CoSyn contributes 112K plotting snippets.Standalone blocks are reconstructed with mock or inline data so they can execute independently.Source: §3.1 · p. 3
  • Runtime checks retain only scripts that execute successfully and generate valid image files, yielding 105K validated stack-edu scripts and 50K CoSyn scripts.Execution occurs in isolated Jupyter environments with timeouts and errors disallowed.Source: §3.1 · p. 4
  • GPT-4o generates instructions from each validated code block and rendered plot, covering setup, data, input preview, plot structure, and visual style.The fixed template provides consistent descriptions of the target plot, data, and rendering requirements.Source: §3.1 · p. 4

3.2 Multi-turn Instruction-following Dialogues with Execution Feedback

  • The correction component contains 45K Python dialogues selected from 56K after removing excessively long or numerous interactions.These dialogues contain model code, execution feedback, and revision prompts, teaching runtime-guided correction despite not being visualization-specific.Source: §3.2 · p. 4

4 Experiment Setup

VisCoder is trained at two parameter scales and evaluated on executable, semantically accurate visualization generation across three plotting libraries. The evaluation adds execution-pass and self-debug protocols to measure both initial validity and recovery from failed code.

Training Setup

  • Qwen2.5-Coder-Instruct is fine-tuned at 3B and 7B scales for three epochs using full-parameter training.Training uses a 5 × 10^-6 learning rate, bfloat16 precision, 8×A100 GPUs, and a total batch size of 128.Source: §4 · p. 4

Evaluation Setup

  • PandasPlotBench contains 175 tasks across matplotlib, seaborn, and plotly, requiring executable and semantically accurate code from instructions and DataFrame previews.Its metrics include Incorrect Code Rate, task-based alignment, and visual similarity scores.Source: §4 · p. 4
  • Execution Pass Rate is added because Incorrect Code Rate can classify blank or semantically meaningless rendered outputs as correct.The new metric measures the percentage of outputs that execute without error.Source: §4 · pp. 4–5

Self-Debug Evaluation Mode

  • Self-debugging retries only initially failed tasks for up to K rounds using dialogue history, failed code, and execution-error feedback.A task is removed from later rounds once revised code executes without error and produces a valid plot.Source: §4 · p. 5
  • The protocol records either the first successful revision or the final failed attempt and evaluates all tasks using the same scoring pipeline.This developer-style loop is summarized in Algorithm 1.Source: §4 · p. 5

5 Main Results

VisCoder improves executable visualization generation over untuned and open-source baselines, with gains from self-debugging and complementary training data. Results also show persistent semantic-error limitations and that Plotly remains especially challenging.

5.1 Overall Model Comparison

  • VisCoder consistently outperforms untuned Qwen2.5-Coder models, surpasses GPT-4o-mini on Seaborn and Plotly at 7B, and remains slightly behind GPT-4o on Matplotlib.The benchmark compares execution success, task alignment, and visual fidelity across three plotting libraries.Source: §5.1 · pp. 5–6, Table 1 · p. 6

5.2 Self-Debug Evaluation Results

  • Self-debugging steadily raises execution pass rates, with VisCoder-7B exceeding 90% on Matplotlib and Seaborn and approaching GPT-4o on Seaborn.The largest improvement typically occurs after the first correction round, while later rounds provide smaller gains.Source: §5.1 · p. 6, §5.2 · pp. 6–7, Figure 2 · p. 7

5.3 Error Analysis

  • Structural errors are corrected effectively, including Seaborn AttributeErrors falling from 15 to 2 and Plotly TypeErrors from 3 to 1.These errors usually have clear diagnostics and involve incorrect method calls, argument types, or simple syntax mistakes.Source: §5.3 · p. 7, Table 2 · p. 7
  • Semantic errors remain difficult: Plotly ValueErrors decline from 29 to 23, while KeyErrors remain at 1 because correction requires reasoning over data structures.The model does not dynamically reassess the DataFrame during self-debugging, so retries can preserve faulty assumptions.Source: §5.3 · p. 7

5.4 Training Data Ablation

  • Code-Feedback improves default execution by 9.7 points on Matplotlib and 14.9 points on Plotly, but Seaborn remains low at 44.0 and below the full model.Execution-feedback examples improve structural validity, but their general-purpose scope lacks visualization-specific breadth.Source: §5.4 · p. 8
  • The full VisCode-200K dataset provides the most consistent gains, whereas Stack-Edu and CoSyn subsets generalize poorly across libraries and modes.CoSyn pass rates remain near zero, while Stack-Edu self-debug scores improve over defaults but remain below the untuned baseline.Source: §5.4 · pp. 7–8

6 Conclusion

VisCode-200K combines executable visualization examples with runtime-feedback correction dialogues, while VisCoder improves over open-source baselines and narrows the gap to proprietary models. The paper emphasizes domain-specific tuning and multi-turn correction as directions for robust, semantically grounded visualization models.

6 Conclusion

  • VisCode-200K combines executable plotting examples with multi-turn correction dialogues grounded in runtime feedback.The dataset supports Python visualization code generation and iterative correction.Source: §6 · p. 8
  • VisCoder substantially outperforms strong open-source baselines across execution and alignment metrics and narrows the gap to GPT-4o-mini.Gains are particularly pronounced for complex visualization structures such as Plotly and iterative self-debugging.Source: §6 · p. 8
  • The paper identifies domain-specific instruction tuning and multi-turn correction supervision as important for robust, semantically grounded visualization-capable models.Future extensions include broader plotting libraries, richer correction supervision, and evaluations of execution-error recovery.Source: §6 · p. 8

Limitations

VisCoder’s scope is limited to Python, while its Plotly performance remains comparatively weaker and its automatic evaluation lacks independent bias or reliability analysis. The appendix describes prompt templates for constructing executable visualization examples and structured instructions from code, data, and rendered plots.

Limitations

  • VisCoder is limited to Python, leaving visualization tasks in languages such as R and JavaScript unexplored.This is the clearest cross-language scope boundary identified by the paper.Source: Limitations · p. 9
  • Plotly remains comparatively weaker because its verbose syntax and complex API frequently produce semantic execution errors that self-debugging struggles to resolve.The evaluation also relies on an automatic judge model without independent analysis of potential bias or reliability.Source: Limitations · p. 9

A Prompts Used for Dataset Construction

  • Dataset-construction prompts extract standalone runnable Python visualization code and generate instructions aligned with rendered plots.The prompts specify setup, data, plot structure, and styling, with variants including copied data-generation lines for mock-data examples.Source: A Prompts Used for Dataset Construction · p. 12
  • The construction prompts require plain-text numbered task parts describing libraries, data, plot structure, and visual styling.The extraction prompt uses mock data when needed and returns null when a meaningful runnable block cannot be constructed.Source: A Prompts Used for Dataset Construction · p. 12

B Breakdown Results in Self-Debug Mode Evaluation

The self-debug evaluation reports execution pass rates by plotting library across up to three automatic correction rounds. Tables separately cover Matplotlib, Seaborn, and Plotly under normal and self-debug settings.

B Breakdown Results in Self-Debug Mode Evaluation

  • Self-debug breakdowns report execution pass rates across up to three automatic correction rounds, grouped by model series and visualization library.The evaluation compares normal execution with iterative correction.Source: B Breakdown Results in Self-Debug Mode… · p. 13

B.1 Matplotlib

  • Matplotlib results compare execution pass rates under normal and self-debug settings after initially failed models receive up to three correction rounds.The table reports percentages for Matplotlib tasks.Source: Table 4 · p. 13

B.2 Seaborn

  • Seaborn results report execution pass rates with up to three automatic correction rounds after initial failure.The table focuses on Seaborn tasks and compares models under the self-debug protocol.Source: Table 5 · p. 13

B.3 Plotly

  • Plotly results report execution pass rates under normal and self-debug settings with up to three automatic correction rounds.The table isolates Plotly tasks for library-specific comparison.Source: Table 6 · p. 14

C Breakdown Results by Error Type

The error-type breakdown counts Python exceptions across model families, plotting libraries, and self-debugging rounds. Separate tables cover VisCoder, GPT, Qwen2.5, and LLaMA series, while the supplied passages provide no cell-level comparisons beyond the stated table scopes.

C Breakdown Results by Error Type

  • Error analysis reports Python exception counts under default execution and across up to three automatic correction rounds.Breakdowns are organized by model family, plotting library, and self-debugging round.Source: C Breakdown Results by Error Type · p. 15

C.1 VisCoder Series

  • VisCoder error distributions are tabulated across Matplotlib, Seaborn, and Plotly at different self-debugging rounds after initial failure.Separate tables cover VisCoder-7B and VisCoder-3B.Source: Table 7 · p. 15, Table 8 · p. 15

C.2 GPT Series

  • GPT error distributions are tabulated across Matplotlib, Seaborn, and Plotly at different self-debugging rounds after initial failure.The supplied table covers GPT-4o.Source: Table 9 · p. 16, Table 10 · p. 16

C.3 Qwen2.5 Series

  • Qwen2.5 error distributions are tabulated across Matplotlib, Seaborn, and Plotly for coder and non-coder model variants.The supplied tables cover Qwen2.5-Coder-7B-Instruct, Qwen2.5-7B-Instruct, Qwen2.5-Coder-3B-Instruct, and Qwen2.5-3B-Instruct.Source: Table 11 · p. 17, Table 12 · p. 17, Table 13 · p. 18

C.4 LLaMA Series

  • LLaMA error distributions are tabulated across Matplotlib, Seaborn, and Plotly at different self-debugging rounds after initial failure.The supplied tables cover Llama-3.1-8B-Instruct and Llama-3.2-3B-Instruct.Source: Table 15 · p. 19, Table 16 · p. 19

D Extended Results and Analyses

Extended evaluations show that VisCode-200K improves VisCoder across model scales and code-generation backbones. The gains hold across plotting libraries and both default and self-debug settings.

D Extended Results and Analyses

  • The extended analyses evaluate model scaling, additional code-generation backbones, and dataset diversity beyond the main experiments.These analyses are presented as additional results supporting the broader evaluation of VisCode-200K and VisCoder.Source: D Extended Results and Analyses · p. 20

D.1 Model Scale Extension

  • VisCoder consistently outperforms Qwen2.5-Coder baselines at 1.5B, 3B, 7B, and 14B scales across libraries and evaluation settings.All scales use the same VisCode-200K fine-tuning configuration; Table 17 reports execution pass rates with and without self-debugging.Source: §D.1 · p. 20, Table 17 · p. 20

D.2 Baseline Model Extension

  • VisCode-200K provides consistent gains when fine-tuning DeepSeek-Coder, CodeLLaMA, and StarCoder on PandasPlotBench.The extended backbone evaluation compares each fine-tuned model with its respective baseline under the same setup.Source: §D.2 · p. 20, Table 18 · p. 20

D.3 Dataset Diversity Analysis

The dataset analysis characterizes VisCode-200K’s library and visualization-type coverage, while case studies illustrate successful generation and multi-round self-debug recovery across Matplotlib, Seaborn, and Plotly.

D.3 Dataset Diversity Analysis

  • VisCode-200K covers 37 visualization types across 13 categories, while retaining common-library frequencies after bias mitigation and runtime validation.The dataset retains non-Matplotlib samples, downsamples Matplotlib code, and validates remaining samples for executability and renderability.Source: §D.3 · p. 21, Table 20 · p. 21, Table 19 · p. 21

E Case Study

  • Representative VisCoder-7B examples demonstrate model behavior across Matplotlib, Seaborn, and Plotly.The case study presents examples from all three visualization libraries.Source: E Case Study · p. 22

E.1 Matplotlib: Successful Generation

  • Matplotlib generation can execute successfully and produce a plot consistent with the ground truth.Figure 3 presents a successful generation example.Source: Figure 3 · p. 22

E.2 Matplotlib: Self-Debug Recovery

  • A Matplotlib AttributeError is corrected in the first self-debug round, producing a plot matching the intended semantics.Figure 4 illustrates recovery from an initially failed generation.Source: Figure 4 · p. 23

E.3 Seaborn: Successful Generation

  • Seaborn generation can execute successfully and produce a plot consistent with the ground truth.Figure 5 presents a successful Seaborn generation example.Source: Figure 5 · p. 24

E.4 Seaborn: Self-Debug Recovery

  • A Seaborn AttributeError is resolved in the third self-debug round, yielding a corrected plot matching the intended semantics.Figure 6 illustrates multi-round recovery from an initially failed generation.Source: Figure 6 · p. 25

E.5 Plotly: Successful Generation

  • Plotly generation can execute successfully and produce a plot consistent with the ground truth.Figure 7 presents a successful Plotly generation example.Source: Figure 7 · p. 26

E.6 Plotly: Self-Debug Recovery

  • A Plotly ValueError is resolved in the second self-debug round, resulting in a corrected plot matching the intended semantics.Figure 8 illustrates recovery from an initially failed generation.Source: Figure 8 · p. 27

Related papers

Loading 2506.03930v2…