Source-linked AI summary
Chat2VIS: Generating Data Visualisations via Natural Language using ChatGPT, Codex and GPT-3 Large Language Models
Paula Maddigan, Teo Susnjak
TL;DR
NL2VIS seeks to generate visualisations from natural language, but ambiguity, underspecification, and typographical errors make intent difficult to interpret. Chat2VIS uses pre-trained GPT-3, Codex, and ChatGPT models with engineered prompts to generate visualisation code and select chart types. Across six case studies, the results support LLMs as an efficient, reliable, and accurate end-to-end approach, while evaluation remains limited in scope.
Problem
Natural-language visualisation systems must interpret ambiguous, underspecified, and typo-filled queries, making accurate NL2VIS difficult.
Method
Chat2VIS combines pre-trained GPT-3, Codex, and ChatGPT with Description and Code prompts to generate Python visualisation scripts and select chart types.
Results
Six case studies support LLMs as an effective end-to-end NL2VIS solution, with the three evaluated LLMs showing no large performance deviations.
Takeaways & Limitations
Pre-trained LLMs with engineered prompts provide a simpler pathway to robust NL2VIS without hand-crafted grammars or customised domain-specific language models.
Takeaways & Limitations
The evaluation covers a limited number of case studies and queries and does not yet include end-user feedback or qualitative usefulness assessment.
Abstract
from arXiv · showhide
The field of data visualisation has long aimed to devise solutions for generating visualisations directly from natural language text. Research in Natural Language Interfaces (NLIs) has contributed towards the development of such techniques. However, the implementation of workable NLIs has always been challenging due to the inherent ambiguity of natural language, as well as in consequence of unclear and poorly written user queries which pose problems for existing language models in discerning user intent. Instead of pursuing the usual path of developing new iterations of language models, this study uniquely proposes leveraging the advancements in pre-trained large language models (LLMs) such as ChatGPT and GPT-3 to convert free-form natural language directly into code for appropriate visualisations. This paper presents a novel system, Chat2VIS, which takes advantage of the capabilities of LLMs and demonstrates how, with effective prompt engineering, the complex problem of language understanding can be solved more efficiently, resulting in simpler and more accurate end-to-end solutions than prior approaches. Chat2VIS shows that LLMs together with the proposed prompts offer a reliable approach to rendering visualisations from natural language queries, even when queries are highly misspecified and underspecified. This solution also presents a significant reduction in costs for the development of NLI systems, while attaining greater visualisation inference abilities compared to traditional NLP approaches that use hand-crafted grammar rules and tailored models. This study also presents how LLM prompts can be constructed in a way that preserves data security and privacy while being generalisable to different datasets. This work compares the performance of GPT-3, Codex and ChatGPT across a number of case studies and contrasts the performances with prior studies.
1 Introduction
Natural-language visualisation interfaces aim to make data analysis more accessible, but translating ambiguous, underspecified, and error-prone queries into visualisations remains difficult. The paper proposes using pre-trained LLMs and prompt engineering to simplify and improve end-to-end NL2VIS.
- Motivation: Natural-language interfaces let users request visualisations without programming or technical constructs, potentially broadening access to data insights.They are intended to make visualisation tools more convenient, novice-friendly, effective, and inclusive.
- Challenges: NL2VIS requires parsing queries, identifying data attributes and analytic tasks, translating them into executable code, and selecting an appropriate chart.The pipeline includes operations such as filtering, correlation, and trend analysis, and each component is error-prone.
- Challenges: Ambiguity, underspecified requirements, and typographical errors make user intent difficult to interpret and appropriate visualisations difficult to generate.These difficulties persist in existing NL2VIS technologies and approaches.
- Research gap: Existing visualisation NLIs largely depend on NLP models and hand-crafted grammar rules, while larger LLMs such as GPT-3 had not yet been directly explored for visualisation generation.The paper positions pre-trained LLMs as a way to address language understanding and code generation in an end-to-end system.
- Proposed direction: Chat2VIS investigates GPT-3, Codex, and ChatGPT for automatically selecting chart types and generating visualisations from free-form conversational language.The study compares the models across case studies and reports potential gains in accuracy, efficiency, and development cost reduction.
2 Related Work
NL2VIS research has progressed from symbolic grammar-based systems toward neural and end-to-end models, but recent large pre-trained LLMs and delegated chart selection remain underexplored. Chat2VIS addresses this gap by using LLMs to convert free-form queries into visualisation code and select chart types.
- Existing systems: Commercial NLIs such as Watson Analytics, Power BI, Tableau, ThoughtSpot, and Google Spreadsheet allow natural-language visualisation requests, but typically constrain queries and chart types.These systems generally do not support more complex or open-ended visualisation tasks.
- Symbolic approaches: Earlier NL2VIS systems mainly used symbolic NLP, including heuristic, rule-based, and probabilistic grammar approaches with different accuracy, flexibility, and resource requirements.Examples include Articulate, DataTone, Eviza, Deep-Eye, NL4DV, and FlowSense.
- Neural approaches: Neural end-to-end approaches combine language understanding, reasoning, and chart generation in one system to improve robustness, flexibility, and adaptability.ADVISor uses customised deep-learning modules and BERT-based input representations, while chart selection remains rule-based.
- Neural approaches: ncNet uses transformer-based, visualisation-aware optimisation and is trained on nvBench to map natural-language queries to visualisations.It can accept an optional chart template to constrain possible outputs and has been evaluated quantitatively and through a user study.
- Research gap: The literature shows a shift toward transformer-based end-to-end systems, but larger pre-trained LLMs and LLM-driven chart selection remain insufficiently explored.Chat2VIS aims to simplify the NL2VIS pipeline for free-form and complex queries while delegating visualisation-type decisions to the AI component.
- Research aims: The study asks whether current LLMs support accurate end-to-end NL2VIS, how prompting can elicit correct charts, which models are most robust, and what limitations remain.These questions are stated as RQ1 through RQ4.
3 Methodology
Chat2VIS uses selected OpenAI LLMs and engineered prompts to turn free-form queries and dataset metadata into Python visualisations rendered in Streamlit. Its prompts provide structured dataset context while withholding raw values, and guide consistent, executable code generation.
- Model selection: Chat2VIS evaluates GPT-3 text-davinci-003, Codex code-davinci-002, and ChatGPT for generating Python visualisation scripts from natural-language queries.The study examines these models without explicitly directing which graph types to generate.
- System architecture: The system combines a user query with a dataset-specific prompt, submits it to selected LLMs, and renders each returned Python script in Streamlit.Users can select datasets, enter free-form intent, choose LLMs, and view the resulting visualisations through the interface.
- Prompt construction: The Description Prompt uses a Python docstring, while the Code Prompt supplies Python statements, imports, dataframe naming, and plotting guidance.The prompts request a single subplot with a fixed figure size, include Python-version guidance, and conclude with instructions to create a plotting script.
- Dataset representation: The prompt exposes dataframe column names, data types, and selected categorical values while withholding raw dataset values to preserve privacy.Object columns with fewer than 20 distinct values are treated as categorical and enumerated to help the LLM identify request-related keywords.
- API execution: The two prompt elements are amalgamated and sent to the LLMs through the text completion endpoint API.The implementation sets temperature to 0 and limits responses to 500 tokens, with a stopping point to avoid multiple scripts.
- User refinement: Users can edit returned Python code and refine queries to request alternative chart types, colours, or labels.This provides a post-generation path for improving visualisations after each model response.
4 Results
Across six case studies, Chat2VIS generated visualisations that generally matched expected outputs while handling broad, misspelled, complex, and underspecified queries. The results also reveal model-specific differences in filtering, chart choice, ordering, and visual clarity.
- Case Study 1: Department Store Dataset: All three LLMs produced more informative axis labels and titles than the nvBench ground-truth visualisation in the department-store case.
- Case Study 2: Colleges Dataset: GPT-3 generated a scatter plot, Codex a bar chart, and ChatGPT a box-and-whisker plot for the broad colleges query.All models identified public and private colleges through the Control column and inferred the relevant median debt and earnings attributes.
- Case Study 3: Energy Production Dataset: GPT-3 and ChatGPT correctly filtered oil-production data from 2004 onward, whereas Codex and ADVISor included earlier data and NL4DV produced an incorrect visualisation.All three LLMs selected a line plot, but only GPT-3 and ChatGPT incorporated the temporal filter into the generated code.
- Case Study 4: Customers and Products Contacts Dataset: All three models generated visualisations similar to the nvBench ground truth for the extra-hard products query, although Codex used a sub-optimal multiple-bar plotting structure.ChatGPT supplied a more informative title, while all three models improved axis labelling relative to nvBench.
- Case Study 5: Misspelled Prompts: All three LLMs correctly interpreted a query containing multiple typographical errors, with ChatGPT producing a descending rank-ordered bar graph and the other models ordering movies alphabetically.
- Case Study 6: Underspecified and Ambiguous Prompts: Each LLM inferred that the underspecified query “tomatoes” referred to the Rotten Tomatoes Rating column, but their visualisations differed in usefulness.GPT-3 paired the rating with IMDb Rating, Codex produced an overcrowded plot, and ChatGPT generated a meaningful distribution plot.
5 Discussion
Across six case studies, prompted LLMs supported end-to-end NL2VIS generation with efficiency, cost, accuracy, and robustness advantages, but aesthetic control, output variability, prompt tuning, and evaluation scope remained challenges.
- Six case studies confirmed that prompted LLMs can support end-to-end visualisation generation from natural language.The study reports efficiency, cost savings, accuracy, and robustness to misspecified and underspecified requests.
- Description and Code Primer definitions helped prompt LLMs to generate appropriately rendered charts from natural-language requests.The Description specifies underlying data attributes, while the Code Primer provides a coding guide.
- The three LLMs showed no large preliminary performance deviations, likely because they were trained on similar datasets.
- Remaining challenges: Aesthetic styling remained inconsistent: background colours, grid lines, and plot-element colours depended on plot type, rendering environment, or unsuitable parameters.Grid-line refinement was more successful when paired with an explicitly stated plot type.
- Remaining challenges: Repeated prompts could produce substantially different plot types and features, particularly with ChatGPT, because stochastic controls were unavailable.
- Study limitations and future work: The evaluation used a limited set of case studies and queries rather than end-user feedback or a broad quantitative benchmark.Future work proposed nvBench-based analysis across more queries and more robust comparisons with prior studies.
6 Conclusion
Chat2VIS uses pre-trained LLMs and engineered prompts to convert free-form natural language into visualisations. The authors report efficient, reliable, and accurate NL2VIS with automatic chart selection, malformed-query handling, and privacy-aware generalisability.
- NL2VIS remains difficult because accurately understanding natural-language input is challenging.
- Chat2VIS uses pre-trained LLMs to understand queries, generate code, and select appropriate visualisation types automatically.The system examines ChatGPT, GPT-3, and Codex for this end-to-end task.
- The system handled vague and malformed queries while providing an efficient, reliable, and accurate NL2VIS solution.
- The approach is described as data-privacy preserving and security-aware, supporting generalisability across datasets.
- LLMs offer a simpler pathway to robust visualisation interfaces without defining grammars or customised domain-specific language models.