PaperlayerView exact revision on arXiv

Source-linked AI summary

Aligning Text, Code, and Vision: A Multi-Objective Reinforcement Learning Framework for Text-to-Visualization

Mizanur Rahman, Mohammed Saidul Islam, Md Tahmid Rahman Laskar, Shafiq Joty, Enamul Hoque

arXiv:2601.04582v1cs.CL

TL;DR

Text2Vis systems often fail to produce visualizations that are simultaneously executable, semantically aligned, and clear, while SFT cannot directly optimize post-execution quality. RL-Text2Vis uses GRPO with a multi-objective reward for textual accuracy, code executability, and visualization quality, achieving strong benchmark results and generalization.

  • Problem

    Text2Vis models struggle with semantic correctness, code executability, and chart readability because SFT token-level losses cannot optimize post-execution visualization qualities.

  • Method

    RL-Text2Vis trains Qwen2.5 7B and 14B models with GRPO and a multi-objective reward using post-execution feedback.

  • Results

    RL-Text2Vis outperforms supervised, zero-shot, and proprietary baselines, including over 22% relative improvement over GPT-4o in chart clarity and correctness and 97% code executability.

  • Takeaways & Limitations

    Post-execution visual feedback with multi-objective RL supports structured, multimodal reasoning and generalization across visualization benchmarks.

  • Takeaways & Limitations

    The 14B model requires considerably more computation and memory, larger backbones were not trained, and robustness in highly specialized domains remains untested.

Abstract

from arXiv · show

Text-to-Visualization (Text2Vis) systems translate natural language queries over tabular data into concise answers and executable visualizations. While closed-source LLMs generate functional code, the resulting charts often lack semantic alignment and clarity, qualities that can only be assessed post-execution. Open-source models struggle even more, frequently producing non-executable or visually poor outputs. Although supervised fine-tuning can improve code executability, it fails to enhance overall visualization quality, as traditional SFT loss cannot capture post-execution feedback. To address this gap, we propose RL-Text2Vis, the first reinforcement learning framework for Text2Vis generation. Built on Group Relative Policy Optimization (GRPO), our method uses a novel multi-objective reward that jointly optimizes textual accuracy, code validity, and visualization quality using post-execution feedback. By training Qwen2.5 models (7B and 14B), RL-Text2Vis achieves a 22% relative improvement in chart quality over GPT-4o on the Text2Vis benchmark and boosts code execution success from 78% to 97% relative to its zero-shot baseline. Our models significantly outperform strong zero-shot and supervised baselines and also demonstrate robust generalization to out-of-domain datasets like VIS-Eval and NVBench. These results establish GRPO as an effective strategy for structured, multimodal reasoning in visualization generation. We release our code at https://github.com/vis-nlp/RL-Text2Vis.

1 Introduction

Text2Vis systems must produce accurate answers, executable code, and clear, query-aligned visualizations, but existing models and token-level training objectives often fail to ensure these post-execution qualities. RL-Text2Vis addresses this gap with GRPO and a multi-objective reward combining textual, code, and visualization feedback.

  • Text2Vis converts natural-language queries over tabular data into concise answers and accurate visualizations, reducing the need for manual coding.
  • LLM-based systems still produce semantically incorrect answers, non-executable code, and unclear or misaligned charts.
  • Post-execution chart readability and semantic alignment cannot be optimized by traditional SFT losses focused on token-level imitation.
  • RL-Text2Vis uses GRPO with a multi-objective reward that jointly evaluates textual correctness, code validity, and visualization quality.
  • 22% relative improvement over GPT-4o in chart clarity and correctness was achieved across Text2Vis experiments with Qwen2.5 7B and 14B models.
  • 97% code executability and strong generalization to VIS-Eval and NVBench distinguish RL-Text2Vis from supervised, zero-shot, and proprietary baselines.

2 Related Work

Text-to-visualization research has progressed from rigid rule-based systems to flexible LLM prompting, but persistent failures in answers, executability, and chart relevance remain. RL-Text2Vis combines post-execution multimodal feedback with multi-objective reinforcement learning to address these limitations.

  • Early systems such as NL4DV and Advisor used rule-based or template-driven specifications that ensured syntactic correctness but were rigid and difficult to scale.
  • Prompting-based methods including Chat2VIS, LIDA, ChartGPT, ChatVis, Prompt4Vis, and ChartLlama translate natural-language queries into visualization code with greater flexibility.
  • LLM-based approaches still generate incorrect answers, non-executable code, or charts lacking clarity and relevance to analytical intent.
  • Prior RL methods such as DeepSeek-R1 and SWE-RL use single-modal signals including code execution success or numeric correctness.
  • RL-Text2Vis introduces post-execution multimodal feedback and a multi-objective reward covering semantic alignment, code validity, and chart readability.

3 RL-TEXT2VIS

RL-Text2Vis applies GRPO to Text2Vis generation, using post-execution feedback and multi-objective rewards to jointly optimize answer, code, and visualization quality. Its pipeline combines structural validation, multimodal quality scoring, and critic-free policy updates.

  • Framework overview: RL-Text2Vis maps a query and table to a concise answer and visualization code, then scores the output after code execution.The reward mechanism combines structural and quality-based signals.
  • GRPO optimization: GRPO generates multiple candidates per prompt and assigns relative advantages from within-group standardized rewards.Outputs above the group average receive positive advantages, while lower-scoring outputs receive negative advantages.
  • GRPO optimization: GRPO stabilizes policy updates with PPO-style ratio clipping and KL regularization against a frozen reference policy.The method avoids training a learned value function.
  • Reward design: The two-stage reward first enforces a valid JSON schema with answer and runnable code fields, then evaluates textual, code, and visual quality.The code must be a Python visualization script ending with plt.show().
  • Reward design: The composite reward is a weighted sum of textual, code, and visual rewards, with the selected weights set to (α, β, γ) = (0.50, 0.25, 0.25).This setting improved code executability and visualization quality without degrading textual correctness.
  • Training setup: The framework trains Qwen2.5-7B and Qwen2.5-14B models on Text2Vis test1, reserving 236 test2 samples for final evaluation.Training uses group-based reward normalization and the two-stage reward scheme.

4 Experiments

The experiments compare RL-Text2Vis with proprietary, open-source, code-specialized, and supervised-fine-tuned baselines under a standardized evaluation protocol. Evaluation covers multiple Text2Vis quality dimensions and includes held-out in-domain and out-of-domain settings.

  • Benchmarks: The study evaluates Text2Vis in-domain and VisEval out-of-domain, with Text2Vis test2 used strictly as a held-out set of 236 stratified samples.VisEval contains 2,524 queries across 146 databases.
  • Baselines: The baseline suite includes GPT-4o, Gemini 1.5 Flash, Gemini 2.0 Flash, LLaMA-3.1-8B, CodeLLaMA, Mistral-7B, Qwen2.5, and supervised variants.The comparison spans closed-source and open-source systems, code-specialized models, and SFT baselines.
  • Experimental settings: GRPO was selected over PPO because preliminary experiments found it more efficient without a value model and more effective than PPO.All systems use temperature = 0.7, top-p = 0.9, and a 2,048-token limit for comparability.
  • Evaluation: The evaluation protocol measures Answer Correctness, Code Executability, Chart Readability, and Chart Correctness.GPT-4o serves as the primary evaluator, supported by reported agreement with human ratings.

5 Results

RL-Text2Vis improves in-domain visualization quality, code execution, and answer matching over open-source and proprietary baselines. It also generalizes across out-of-domain benchmarks, model scales, and architectures, with multimodal rewards producing the strongest overall performance.

  • Readability increased from 3.12 to 4.10 and correctness from 2.94 to 4.03 for RL-Text2Vis 14B versus zero-shot Qwen2.5-14B.
  • Code execution success rose from 78% to 97%, while answer match improved from 29% to 35% on Text2Vis.
  • RL-Text2Vis 14B exceeded GPT-4o in readability, scoring 4.10 versus 3.32, and correctness, scoring 4.03 versus 3.30.
  • On VIS-Eval, RL-Text2Vis-7B improved readability from 1.50 to 2.50, correctness from 0.69 to 1.37, and execution success from 57% to 72%.
  • Human and automated judgments correlated strongly, with r = 0.91 for Answer Match and r = 0.88 for both Clarity & Readability and Chart Correctness.
  • Multimodal rewards achieved the highest overall ablation performance, while sampling eight completions stabilized ranking-based updates and improved chart reliability.
  • RL improved metrics across model scales and families; Qwen2.5-3B execution rose from 67% to 88%, but textual answer accuracy remained unchanged.

6 Error Analysis

Error analysis identifies failures in code execution, numerical and logical correctness, and visualization design. GRPO-based reinforcement learning mitigates these problems through post-execution, multi-objective feedback.

  • Baseline failures included syntax errors, shape mismatches, incorrect calculations, missing imports, logic inconsistencies, and glyph warnings.
  • Visual failures included poor readability and layout, missing axis labels, and weak alignment with query intent.
  • GRPO eliminated syntax and structural errors while improving readability, visual clarity, semantic alignment, labels, and interpretability.

7 Conclusion

RL-Text2Vis introduces a GRPO-based framework that uses post-execution multimodal feedback to jointly optimize textual, code, and visualization objectives. Experiments show consistent gains across models and benchmarks, matching or surpassing GPT-4o within the evaluated scope.

  • RL-Text2Vis is presented as the first reinforcement learning framework for text-to-visualization generation.
  • GRPO with a multi-objective reward jointly optimizes textual accuracy, code executability, and visualization quality using post-execution feedback.
  • Experiments show consistent gains across models and benchmarks, demonstrating improved structured and multimodal reasoning.
  • The framework matches or surpasses proprietary models like GPT-4o while remaining open, efficient, and privacy-preserving.

Ethical Considerations

The paper identifies ethical and operational risks for real-world deployment, including bias, privacy, computational cost, and misuse. It recommends safeguards such as anonymization and human oversight.

  • Visualizations may amplify dataset or query biases and cause misinterpretation in sensitive domains such as healthcare or finance.
  • Real-world deployments require data anonymization and human oversight, despite benchmarks using synthetic or public datasets.
  • RL computational costs raise environmental concerns, while safeguards are needed against deceptive visualizations and adversarial use.

Limitations

RL-Text2Vis has four stated limitations: computational cost, untested larger scales and specialized domains, and a focus on static visualizations.

  • The 14B model achieves the best quality but requires considerably more computation and memory than the 7B variant.The authors describe 7B as the more practical and cost-effective choice for resource-constrained organizations.
  • Larger 32B and 72B backbones were not trained because of resource constraints, although scaling from 3B to 14B showed consistent gains.Further improvements at larger scales remain a proposed future direction.
  • Robustness in highly specialized domains such as medical or financial visualization remains untested.
  • The study covers static visualizations, leaving interactive and multiview visual analytics for future work.

A.1 Reinforcement Learning for LLM Alignment

This section reviews RL alignment methods and describes how GRPO is applied to Text2Vis through group-based rewards, multimodal evaluators, and post-execution assessment. It also summarizes training dynamics, evaluation inputs, and qualitative error improvements.

  • Reinforcement Learning for LLM Alignment: RL learns policies from reward feedback rather than explicit ground-truth labels and has been applied to aligning LLMs with human preferences.
  • Reinforcement Learning for LLM Alignment: RLHF combines supervised fine-tuning, reward-model training from human comparisons, and policy optimization, but is computationally expensive.
  • Proximal Policy Optimization: PPO stabilizes policy updates with a clipped surrogate objective, a critic-based advantage estimate, and KL regularization.
  • Direct Preference Optimization: DPO directly optimizes pairwise preference data without a reward model or reinforcement-learning loop.
  • Group Relative Policy Optimization: GRPO generates groups of candidate outputs and standardizes their rewards within each group to compute advantages without a learned critic.
  • Group Relative Policy Optimization: GRPO retains PPO-style clipping and KL regularization, making it suitable for post-execution metrics such as textual correctness, code executability, and visualization clarity.
  • In-loop Evaluators: Qwen2.5 and Qwen2.5-VL models serve as open-source text and visual judges for the 7B and 14B policies.
Loading 2601.04582v1…