VisCoder2: Building Multi-Language Visualization Coding Agents

• Existing datasets and benchmarks emphasize single-language, single-round generation, contain unreliable snippets, and lack validated execution data or systematic multi-round repair evaluation (Ni et al., 2025; Yang et al., 2023).These limitations prevent evaluation from capturing the complexity of real-world visualization development (Goswami et al., 2025).Source: §1 · p. 2
• VisCode-Multi-679K provides 679K executable visualization and code-correction samples across twelve programming languages, combining validated repository code with multi-turn feedback dialogues.The dataset is designed to train models to revise faulty code based on execution feedback (Zheng et al., 2024).Source: §1 · p. 2
• VisPlotBench evaluates visualization coding agents across eight languages using executable tasks, rendered outputs, and protocols for initial generation and multi-round self-debug.Its natural-language tasks and standardized evaluation target systematic assessment across languages and repair scenarios (Galimzyanov et al., 2024).Source: §1 · p. 2
• 82.4% overall execution pass rate at 32B scale makes VisCoder2 on par with GPT-4.1 and surpasses GPT-4.1-mini, especially for LilyPond, LaTeX, and Asymptote.VisCoder2 also substantially outperforms size-matched open-source baselines (Hui et al., 2024; Guo et al., 2024a; Ni et al., 2025) and closes much of the gap with proprietary models such as GPT-4.1 (Fachada et al., 2025).Source: §1 · p. 2

LLMs for Visualization Code Generation

• Existing visualization code generators mainly target Python, Vega-Lite, or HTML, typically support only one or two languages, and often produce code that fails to run reliably.They also lack systematic execution validation, limiting reliability in practical workflows.Source: §2 · pp. 2–3
• VisCode-Multi-679K provides validated, executable visualization samples across twelve programming languages, including specialized domains such as LaTeX mathematical plots and LilyPond musical notation.This addresses the limited visualization-specific knowledge and quality control of broader multi-language code-generation resources.Source: §2 · p. 3

Self-Debug and Coding Agents

• General coding agents use execution traces, error messages, and runtime outcomes for iterative correction, but visualization systems remain underexplored and lack systematic multi-turn cross-language debugging.VisCoder2 combines multi-language visualization generation with self-debug, particularly supporting symbolic languages where execution validation is essential.Source: §2 · p. 3

Visualization Benchmark

• Existing visualization benchmarks emphasize Python or declarative specifications and mostly evaluate single-turn generation, preventing assessment of diverse toolchains and iterative debugging workflows.VisPlotBench evaluates both initial generation and multi-turn self-debug across visualization tasks in multiple programming languages.Source: §2 · p. 3

• VisCode-Multi-679K combines executable visualization code across twelve languages with 66K multi-turn Code-Feedback dialogues for generation and self-correction.The code samples span diverse chart types, libraries, and real-world usage patterns, while the dialogues supervise revision based on execution feedback.Source: §3 · p. 4

3.1 CODE EXTRACTION FROM PUBLIC REPOSITORIES

• The construction pipeline applies library-based filtering, code-block extraction, runtime validation, and instruction generation to produce unified supervision.Figure 2 summarizes this four-stage process, which collects code, validates execution, renders outputs, and pairs examples with instructions.Source: Figure 2 · p. 3, §3 · p. 4
• The dataset integrates the-stack-v2, svg-diagrams, and CoSyn-400K to cover realistic, domain-specific, and synthetic visualization code across languages and styles.These sources provide complementary coverage of general programs, SVG diagrams, and cleanly structured multi-language snippets.Source: §3.1 · p. 4
• Approximately 5.3M visualization candidates are identified from six languages in filtered the-stack-v2 subsets and reduced to roughly 900K standalone candidate blocks.GPT-4.1-mini extracts plotting blocks from broader programs and injects mock inputs when needed for isolated execution.Source: §3.1 · p. 4
• CoSyn-400K contributes 408K visualization snippets across eight languages, while svg-diagrams retains about 79K candidate blocks after removing incomplete SVG data.CoSyn snippets may require reconstruction of data-loading logic and plotting calls before runtime validation.Source: §3.1 · p. 4
• 245K, 43K, and 322K validated plotting scripts are retained from the-stack-v2, svg-diagrams, and CoSyn-400K, respectively, each paired with a rendered output.Blocks are executed in isolated Jupyter environments with strict error handling, timeouts, and image-quality checks.Source: §3.1 · p. 5
• GPT-4.1-generated instructions describe language and libraries, data or visible elements, output concepts, and visual style in a fixed template.This structure aligns code syntax with visualization semantics and enforces consistency across sources and languages.Source: §3.1 · p. 5

3.2 MULTI-TURN INSTRUCTION-FOLLOWING DIALOGUES WITH EXECUTION FEEDBACK

• The Code-Feedback dialogues cover Python, HTML, JavaScript, R, and other languages, pairing instructions, generated code, execution feedback, and revision prompts.Although not visualization-specific, they teach models to revise faulty code from runtime signals and practice multi-turn refinement alongside single-turn samples.Source: §3.2 · pp. 5–6

4.1 OVERVIEW

• VisPlotBench spans eight languages and 888 tasks across 13 visual categories, enabling systematic evaluation of visualization coding agents across languages and task types.Each task combines a natural-language instruction, executable code, and a rendered output, including underrepresented categories such as Hierarchies, Music, and Networks & Flows.Source: §4.1 · pp. 6–7, Figure 3 · p. 6

4.2 DATA COLLECTION AND CURATION

• The benchmark contains 888 executable tasks across eight languages, 13 visual categories, and 116 subtypes assembled from public examples, documentation, and high-quality code.Candidate scripts are retained only after successful isolated execution and valid rendering, followed by filtering of trivial or duplicate outputs and double-pass annotation review.Source: §4.2 · p. 7

4.3 TASK CONSTRUCTION

• Each task uses a five-part instruction schema—Setup, Plot Instruct, Data Instruct, Task Description, and Style Description—to unify semantic and stylistic requirements across languages.Language-specific components capture syntax, runtime, and data-access conventions, while GPT-4.1 generates semantic and perceptual descriptions from verified code and rendered visuals.Source: §4.3 · p. 7

4.4 EVALUATION PROTOCOL

• The standardized execute–render–score pipeline produces a rendered image, execution log, and metadata record from isolated, time-limited submissions.This supports execution-grounded evaluation alongside judgment-based assessment.Source: §4.4 · pp. 7–8
• Evaluation measures Execution Pass Rate, Task Score, and Visual Score, while a three-round self-debug protocol evaluates feedback-driven recovery from unresolved tasks.Revisions receive the instruction, prior code, and an execution-log excerpt, with the final score taken from the best attempt.Source: §4.4 · p. 8

Training Setup

• Qwen2.5-Coder-Instruct (Hui et al., 2024) is fully fine-tuned at 3B, 7B, 14B, and 32B scales for three epochs using standardized optimization and hardware settings.Training uses a 5 × 10^-6 learning rate, 0.05 warm-up ratio, cosine scheduling, bfloat16 precision, 8×H100 GPUs, batch size 64, and SWIFT (Zhao et al., 2024).Source: §5 · p. 8

Evaluation Setup

• VisPlotBench evaluation reports Execution Pass Rate, Task Score, and Visual Score for baseline generation and up to three rounds of self-debugging from execution feedback.These metrics capture executability, semantic alignment, and perceptual similarity, while self-debugging measures iterative recovery.Source: §5 · p. 8

6.1 OVERALL COMPARISON

• At 32B, VisCoder2 improves overall execution pass rate by approximately 15 points over Qwen2.5-Coder and reaches parity with GPT-4.1.Across scales, VisCoder2 consistently outperforms size-matched open-source baselines, except on SVG, where it trails the strongest baseline by over 10 points.Source: §6.1 · p. 9, Table 3 · p. 9
• Self-debug raises larger-scale overall execution by nearly ten points, especially for LilyPond, LaTeX, and Asymptote, by repairing frequent syntax and compilation failures.GPT-4.1 improves from 31.3% to 66.1% execution pass rate on LaTeX with Self-Debug, while VisCoder2 shows the same improvement trend.Source: §6.1 · p. 9, §6.2 · p. 10

6.2 TASK AND VISUAL SCORE ANALYSIS

• VisCoder2 reaches 69.1% execution on LilyPond at 7B versus Qwen2.5-Coder’s 5.5%, while examples scoring above 75 increase more than tenfold.LaTeX exposes execution–semantics mismatch, whereas SVG shows high execution but substantially lower visual scores, including 95.4% execution and below-50 average visual score for GPT-4.1 with Self-Debug.Source: §6.2 · p. 10

6.3 ERROR ANALYSIS

• Self-debug reduces structural and interface errors, including Python interface errors from 13 to 3 and LilyPond structural errors from 14 to 10.Diagnostic traces make these shallow failures relatively easy to fix, while semantic, runtime, renderer, and environment errors often persist.Source: §6.3 · pp. 10–11, Table 5 · p. 10

6.4 TRAINING DATA ABLATION

• The full VisCode-Multi-679K dataset reaches 70.9% default and 76.4% self-debug overall pass rates, surpassing the untuned baseline and every single-source variant.CoSyn raises LilyPond and Asymptote execution by over 60 points, while Code-Feedback lifts overall execution from 55.2% to 63.1% under self-debug.Source: §6.4 · p. 11, Table 6 · p. 11

• VisCode-Multi-679K, VisPlotBench, and VisCoder2 together provide a systematic framework for building and evaluating multi-language visualization coding agents.The dataset covers executable code and multi-turn feedback in twelve languages, while the benchmark standardizes execute–render–score evaluation across eight languages, 13 categories, and 116 subtypes.Source: §7 · p. 12
• VisCoder2 consistently outperforms open-source baselines and approaches proprietary models in execution reliability.The model family was trained on VisCode-Multi-679K and evaluated through the resources introduced in the paper.Source: §7 · p. 12
• Broad multi-language coverage is essential for generalization, particularly because symbolic and compiler-dependent languages such as LaTeX, LilyPond, and Asymptote remain challenging.Progress on these languages is described as decisive for true generalization.Source: §7 · p. 12
• Iterative self-debugging delivers large gains across models, especially where structural and semantic errors are common.The conclusion identifies refinement based on execution feedback as indispensable for practical visualization coding agents.Source: §7 · p. 12

LIMITATIONS

• Training data is imbalanced across languages, with Python and Vega-Lite better represented than symbolic and domain-specific languages.This imbalance may bias models toward dominant languages.Source: LIMITATIONS · p. 12
• Symbolic and domain-specific languages have substantially fewer training samples than high-resource ecosystems.The limitation concerns uneven corpus representation across language types.Source: LIMITATIONS · p. 12
• Corpus imbalance may bias models toward dominant languages.The passage identifies this as a consequence of uneven language coverage.Source: LIMITATIONS · p. 12
• VisPlotBench currently covers eight visualization languages.Its present scope limits the breadth of multi-language evaluation.Source: LIMITATIONS · p. 12
• Extending VisPlotBench to additional frameworks and languages would broaden coverage.The passage presents this extension as a way to enable more comprehensive evaluation.Source: LIMITATIONS · p. 12

A.1 PROMPT USED IN VISCODE-MULTI-679K

• VisCode-Multi-679K extraction prompts require minimal, standalone visualization snippets that preserve visual intent and style, create meaningful mock data when needed, and return null when rendering information is insufficient.The prompts cover Python, JavaScript, TypeScript, C++, R, and HTML, with language-specific file, dependency, execution, and rendering constraints.Source: §A.1 · p. 18
• VisCode-Multi-679K description prompts infer visual purpose, structure, data or visible content, output, and style from code and rendered images using a fixed five-part plain-text format.For data-driven languages, the format includes verbatim data-generation lines; responses must be numbered and contain no markdown.Source: §A.1 · p. 19
• Separate VisCode-Multi-679K prompt variants use four-part descriptions for data-driven languages and none-data-driven languages, replacing data descriptions with visual-element summaries when appropriate.The data-driven variant covers Python, Vega-Lite, HTML, LilyPond, and Mermaid, whereas the none-data-driven variant covers Asymptote and SVG.Source: §A.1 · pp. 19–20

A.2 PROMPT USED IN VISPLOTBENCH

• VisPlotBench generates task and style descriptions from code and rendered images, requiring a two-part response that separately states the task and visual style without library names or specification keywords.The prompt uses GPT-4.1 across languages including Python, Vega-Lite, HTML, LilyPond, Mermaid, Asymptote, and SVG.Source: §A.2 · p. 21
• VisPlotBench evaluates generated plots with separate visual-correctness and task-adherence judges, scoring each from 0 to 100 and requiring a final score token.The visual judge compares generated and ground-truth figures while ignoring color differences when the conveyed information matches.Source: §A.2 · p. 21

A.3 INSTRUCT DESIGN IN VISPLOTBENCH EVALUATION

• VisPlotBench evaluation instructions constrain each language to one complete, directly renderable artifact, with language-specific syntax, imports or libraries, data handling, and restrictions on explanations or external resources.The specifications cover Python, Vega-Lite, Mermaid, LilyPond, SVG, Asymptote, LaTeX, and HTML; Python and Vega-Lite additionally require preserving dataset structure without destructive column changes.Source: §A.3 · pp. 22–24

B TAXONOMY OF VISUAL CATEGORIES AND SUBTYPES

• VisPlotBench tasks are organized into 13 visual categories and 116 subtypes, covering common and underexplored visualization families.The taxonomy distribution is presented in Figure 4 and summarized in Table 7.Source: Figure 4 · p. 26, Table 7 · p. 27

C BREAKDOWN MAIN RESULTS

• The breakdown evaluates each visualization language using execution pass rate, mean visual and task scores, and the proportion of samples scoring at least 75.These metrics are labeled Exec Pass, Mean, and Good, respectively.Source: C BREAKDOWN MAIN RESULTS · p. 28
• Across the language-specific breakdown, the reported results jointly capture executable success, average visual and task quality, and high-quality sample coverage.The Good metric specifically measures the share of samples scoring at least 75.Source: C BREAKDOWN MAIN RESULTS · p. 28, Table 8 · p. 28, Table 9 · p. 29

C.1 PYTHON, VEGA-LITE & LILYPOND

• Table 8 reports VisPlotBench performance for Python, Vega-Lite, and LilyPond across the Exec Pass, Mean, and Good metrics.Within each model scale, bold and underlined formatting identifies the best and second-best models.Source: Table 8 · p. 28

C BREAKDOWN MAIN RESULTS

• The three tables compare selected languages while preserving model-scale rankings through best-versus-second-best annotations.These annotations are defined consistently across Tables 8–10.Source: Table 8 · p. 28, Table 9 · p. 29, Table 10 · p. 30

C.2 MERMAID, SVG & LATEX

• Table 9 reports VisPlotBench performance for Mermaid, SVG, and LaTeX using execution, mean-score, and threshold-success metrics.The table marks the best-performing model in each scale in bold and the second best with underlining.Source: Table 9 · p. 29

C.3 ASYMPTOTE & HTML

• Table 10 reports VisPlotBench performance for Asymptote and HTML using Exec Pass, Mean, and Good.The table compares models by scale, highlighting the best result in bold and the second-best result with an underline.Source: Table 10 · p. 30

D BREAKDOWN SELF-DEBUG RESULTS

• The breakdown reports execution pass rates for each language across up to three automatic self-debug correction rounds, grouped by model series.The section evaluates performance under the self-debug setting rather than only initial generation.Source: D BREAKDOWN SELF-DEBUG RESULTS · p. 31

D.1 PYTHON & VEGA-LITE

• Table 11 reports Python and Vega-Lite execution pass rates under both normal and self-debug settings, with up to three correction rounds for initially failing models.Python results appear in the left columns and Vega-Lite results in the right columns.Source: Table 11 · p. 31

D BREAKDOWN SELF-DEBUG RESULTS

• Across all language pairs, the tables use the same comparison structure: normal execution versus self-debug performance, including correction rounds for models that initially fail.This shared protocol enables the language-specific breakdowns to be read consistently across the four table groups.Source: Table 11 · p. 31, Table 12 · p. 32, Table 13 · p. 33

D.2 LILYPOND & MERMAID

• Table 12 reports LilyPond and Mermaid execution pass rates under both normal and self-debug settings, allowing up to three correction rounds after initial failure.LilyPond results appear in the left columns and Mermaid results in the right columns.Source: Table 12 · p. 32

D.3 SVG & LATEX

• Table 13 reports SVG and LaTeX execution pass rates under both normal and self-debug settings, with up to three automatic correction rounds.SVG results appear in the left columns and LaTeX results in the right columns.Source: Table 13 · p. 33

D.4 ASYMPTOTE & HTML

• Table 14 reports Asymptote and HTML execution pass rates under both normal and self-debug settings, with up to three automatic correction rounds for initially failing models.Asymptote results appear in the left columns and HTML results in the right columns.Source: Table 14 · p. 34

E BREAKDOWN ERROR TYPE RESULTS

• Execution-error distributions for GPT-4.1 and VisCoder2-32B are reported across up to three automatic-correction rounds for eight visualization languages.The breakdown covers Python, Vega-Lite, Lilypond, Mermaid, SVG, LaTeX, Asymptote, and HTML.Source: E BREAKDOWN ERROR TYPE RESULTS · p. 35, Table 15 · p. 35, Table 16 · p. 35

E.6 LATEX / E.7 ASYMPTOTE / E.8 HTML

• The breakdown additionally covers execution-error distributions for VisCoder2-32B and GPT-4.1 in LaTeX, Asymptote, and HTML across self-debugging rounds.These language-specific tables report error counts at different rounds after initial failure.Source: Table 20 · p. 36, Table 21 · p. 37, Table 22 · p. 37

F ADDITIONAL EXPERIMENTAL RESULTS & DISCUSSIONS

• The appendix supplements the main experiments with evaluation settings, additional benchmark results, and analysis of self-debug behavior and Task/Visual Scores.This section organizes the additional experimental results and discussions.Source: F ADDITIONAL EXPERIMENTAL RESULTS &… · p. 38

F.1 EVALUATION SETTINGS

• Self-debug evaluation retries only unresolved tasks for up to K = 3 rounds, using the original instruction, failed code, and execution feedback to generate revised code.Successful tasks are removed from later rounds, while the latest failed output is carried forward and finally evaluated through the same pipeline.Source: §F.1 · p. 38
• Task Score compares the generated plot with the task instruction, whereas Visual Score compares it with the ground-truth image using GPT-based judging.The scoring procedure follows PandasPlotBench (Galimzyanov et al., 2024), with judge prompts provided in Appendix A.2.Source: §F.1 · p. 38

F.2 ADDITIONAL EVALUATION RESULTS

• Across Matplotlib, Seaborn, and Plotly, VisCoder2 consistently outperforms Qwen2.5-Coder on execution success and semantic and perceptual metrics, with self-debug bringing VisCoder2-14B close to GPT-4.1.PandasPlotBench reports execution pass rate, task and visual scores, and the proportion of samples scoring at least 75.Source: §F.2 · p. 38, Table 23 · p. 39
• The additional evaluations support VisCoder2’s generalization to unseen visualization benchmarks while preserving its target gains in cross-language executable visualization and multi-round self-debug.The conclusion combines the PandasPlotBench generalization finding with the HumanEval stability analysis.Source: §F.2 · pp. 38–39
• VisCoder2 shows only a modest 2–3 point decrease versus base Qwen2.5-Coder models on HumanEval and HumanEval+, while maintaining stable general coding ability.The difference is attributed to distributional differences between visualization-code training and HumanEval’s algorithmic and data-structure focus.Source: §F.2 · p. 39, Table 24 · p. 39

F.3 DEEP ANALYSIS OF SELF-DEBUG BEHAVIOR AND TASK/VISUAL SCORE

• Self-debug improves execution reliability across models and most languages, with VisCoder2-32B exceeding the baseline in seven languages and especially benefiting symbolic languages.The first round produces the largest gains, while later rounds typically plateau; feedback is most effective for structural and shallow syntactic errors.Source: §F.3 · pp. 40–41
• Remaining self-debug failures primarily involve deep semantic inconsistencies, grammar- or compiler-dependent symbolic errors, and runtime or rendering behaviors that feedback does not localize.These include incorrect data relationships, macro or scope failures, unsupported APIs, missing packages, infinite loops, malformed SVG, and coarse non-localized diagnostics.Source: §F.3 · pp. 40–41
• Execution, Task Score, and Visual Score rise together in Python, Vega-Lite, Mermaid, and HTML, whereas LaTeX and SVG show clear execution–quality decoupling and Asymptote remains difficult.For example, VisCoder2-32B improves in Python from Visual Score 49 to 58 and Task Score 56 to 68, while GPT-4.1 in Mermaid rises from 41 to 56 visually and 57 to 77 on task.Source: §F.3 · pp. 41–42

F.4 QUANTITATIVE ANALYSIS OF VISCODE-MULTI-679K

• VisCode-Multi-679K covers 91 visualization types grouped into 15 categories, while its measured instruction redundancy is 0.21% and code redundancy is 3.03%.The redundancy analysis normalizes instructions and code separately and uses SHA-1 fingerprints to detect exact duplicates.Source: §F.4 · pp. 43–44

G CASE STUDY

• Case studies across Python, Vega-Lite, LilyPond, Mermaid, SVG, LaTeX, Asymptote, and HTML show successful generations alongside recoveries after one to three self-debug rounds.The examples include corrected ValueError, SyntaxError, TypeError, ExPatError, NameError, and ImportError cases, while corresponding failed cases remain unresolved after three rounds.Source: G CASE STUDY · p. 45, Figure 6 · p. 46, Figure 9 · p. 49

G.1 PYTHON: SUCCESSFUL GENERATION; G.4 VEGA-LITE: SUCCESSFUL GENERATION; G.7 LILYPOND: SUCCESSFUL GENERATION; G.10 MERMAID: SUCCESSFUL GENERATION; G.13 SVG: SUCCESSFUL GENERATION; G.16 LATEX: SUCCESSFUL GENERATION; G.19 ASYMPTOTE: SUCCESSFUL GENERATION; G.22 HTML: SUCCESSFUL GENERATION

• Python, Vega-Lite, Lilypond, Mermaid, SVG, and LaTeX each have successful-generation examples in which generated code executes successfully and produces a plot consistent with the ground truth.Source: Figure 5 · p. 45, Figure 8 · p. 48, Figure 11 · p. 51

G.3 PYTHON: SELF-DEBUG FAILED; G.6 VEGA-LITE: SELF-DEBUG FAILED; G.9 LILYPOND: SELF-DEBUG FAILED; G.12 MERMAID: SELF-DEBUG FAILED; G.15 SVG: SELF-DEBUG FAILED; G.18 LATEX: SELF-DEBUG FAILED; G.21 ASYMPTOTE: SELF-DEBUG FAILED; G.24 HTML: SELF-DEBUG FAILED

• The case studies also document unresolved self-debug failures after three rounds in Python, Vega-Lite, LilyPond, Mermaid, SVG, LaTeX, Asymptote, and HTML.Failed examples include AttributeError, TypeError, SyntaxError, AttributeError, ParseError, NameError, TypeError, and TypeError cases, respectively.Source: Figure 7 · p. 47, Figure 10 · p. 50, Figure 13 · p. 53