A Chat Assistant for Software Exploration in a 3D Software Visualization

I. INTRODUCTION

• LLM-driven approaches have largely focused on 2D visualization, with limited work in 3D software visualization, while traditional program interaction can be frustrating and time-consuming.The authors present textual LLM interfaces as a promising way to improve usability.Source: §I · p. 1
• The assistant is integrated into ExplorViz to answer questions using visualization-state context and trigger frontend changes; an eleven-participant evaluation examines comprehension, tool use, and restructuring.ExplorViz employs the city metaphor.Source: §I · p. 1

II. BACKGROUND

• ExplorViz maps applications to city foundations, packages to stacked districts, and classes to buildings, with labels and optional metric-based building dimensions and filters.Hovering reveals details such as a class’s methods.Source: §II · pp. 1–2
• ExplorViz analyzes software structure and Git history, and displays runtime method calls as yellow arcs between class buildings.Runtime traces are collected through instrumentation and exported to the OpenTelemetry Collector in accordance with the OpenTelemetry standard.Source: §II · p. 2

III. RELATED WORK

• Prior work reports useful LLM-generated code explanations and refactoring suggestions, but also hallucinations, inconsistency, and precision concerns.Tool-augmented visualization agents have also shown variable output quality, motivating guardrails and user confirmation.Source: §III · pp. 2–3
• This approach binds natural-language requests to existing ExplorViz entities and operations, making changes easier to confirm and undo than synthesized 3D representations, while retaining a need for guardrails.The authors relate their design to prior tool-augmented agents and emphasize transparent changes and explicit confirmation.Source: §III · p. 3

IV. CHAT ASSISTANT IMPLEMENTATION

• Because visualization state is managed in the frontend, the system integrates CopilotKit directly into React rather than using a separate MCP server; an AI Chat Service bridges the frontend and selected LLM provider.The service manages API keys outside the browser while forwarding prompts, context, responses, and tool-call instructions.Source: §IV · p. 3
• Context interfaces prioritize relevance to available tools, compact size, and stability across renders, so communication traces are omitted because they can be large and frequently updated.Provided state includes landscape summaries, visibility and highlight status, removed entities, and visualization settings.Source: §IV · p. 4
• Frontend tools query landscape data, control the view, and restructure its displayed state; restructuring changes are logged for undo or redo and can be exported as GitLab issues, not applied to source code.The interface supports chat-history navigation, transcript downloads, provider and model selection, and file uploads.Source: §IV · pp. 4–5

V. VISUALIZATION EXAMPLE

• The example begins with a request to explain the customer service and show it using camera movement, highlighting, and pinging.The chat interface is exposed in the ExplorViz frontend’s right-hand sidebar.Source: §V · p. 5, Figure 3 · p. 5
• The assistant first queries landscape data, generates a service description, and then calls frontend tools to move the camera, ping, and highlight the service.The resulting view focuses on the customer service, colors its city foundation red, and temporarily pings the application center.Source: §V · p. 5

VI. EVALUATION

• The authors conducted an empirical evaluation in December 2025.The supplied passage does not specify participants, tasks, or findings.Source: §VI · p. 5

• The first research question concerns perceived benefits and limitations for program comprehension when a chat assistant can access software visualization data.Source: A. Research Questions · p. 5
• The second asks how natural-language-triggered tool calls compare with manual interface interaction in effort, usability, and perceived correctness.Source: A. Research Questions · p. 5
• The third examines advantages and disadvantages of chat-based restructuring, including alignment with user intent and possible improvements.Source: A. Research Questions · pp. 5–6

• The study took place in the Software Engineering Group’s computer laboratory at Kiel University, using a two-monitor desktop running Debian GNU/Linux 13.Source: B. Setup · p. 6
• The desktop had an Intel Core i5-6500 CPU, 16GB DDR4 RAM, an Nvidia GTX 1070 GPU, and 34-inch and 24-inch monitors.Source: B. Setup · p. 6
• The evaluation used the distributed PetClinic sample application, prepared Docker images for the backend, chat service, and frontend, and Firefox 140.6.0esr.Source: B. Setup · p. 6

• Eleven people, mostly computer science students, took part in the evaluation.Source: C. Participants · p. 6
• Participants rated programming proficiency, ExplorViz familiarity, and LLM-tool usage frequency; Table I reports means, standard deviations, and ranges on a 1–5 scale.Source: C. Participants · p. 6, Table I · p. 6
• All participants had used ChatGPT, while nine reported code generation or refactoring use and eight reported debugging use.Source: C. Participants · p. 6

• The assistant used Claude Sonnet 4.5, selected for compatibility with implemented features and a balance of quality, response time, and token usage.Source: D. Methodology · pp. 6–7
• Before the tasks, participants received study information, gave background data, and completed a tutorial on core ExplorViz features without data collection.Source: D. Methodology · p. 7
• Participants described the PetClinic vets service and a DTO package before requesting assistant summaries or explanations and rating information gain and perceived correctness.Source: D. Methodology · p. 7
• Participants manually highlighted model classes and then asked the assistant to do the same, rating correctness, ease of use, and possible mistakes.Source: D. Methodology · p. 7
• Participants created a color theme, requested variants, and prompted restructuring; they rated expectations and quality, while task and overall feedback used ratings and free-text remarks.Manual restructuring was omitted because it would have been too time consuming for the evaluation.Source: D. Methodology · p. 7

E. Results

• In Task 2, response quality averaged 4.64/5; in Task 3, all participants received additional information and rated answer correctness/quality 4.55/5.All but one participant who did not answer recognized that the Task 2 service managed veterinarian data; all Task 3 participants said the assistant explained DTO, though comments noted correctness was difficult to assess without deep software knowledge.Source: E. Results · pp. 7–8
• All participants preferred chat-assisted highlighting, and inspection of chat logs confirmed that it found all six model classes in PetClinic.Participants rated the highlighting tool actions very good and reported no mistakes.Source: E. Results · p. 8
• Chat-assisted color-theme changes were easy to use, averaging 4.82/5, although two participants found the generated themes unsatisfactory.One participant requested a rainbow and received a blue background with orange shades; another said most colors appeared unchanged.Source: E. Results · p. 8
• In Task 6, only four participants were satisfied or fully satisfied with the visual restructuring assistant’s result, yielding a mean expectation-match score of 3.0/5; comments described off-target changes, confusion, and inconsistent success.Source: E. Results · p. 8
• Explanations averaged 3.73/5 and perceived correctness/response quality averaged 3.18/5 for restructuring; eight of eleven participants needed an additional prompt or confirmation.One participant said the agent needed more information about how to improve metrics.Source: E. Results · pp. 8–9
• Nine of eleven participants rated response time fast or rather fast, and overall experience averaged 4.36/5, with all participants finding the assistant easy to use.Four participants rated the overall experience very good and seven rated it good.Source: E. Results · p. 9
• Chat logs suggest token use was uneven across participants, while Table II reports token usage and resulting cost for Claude Sonnet 4.5.The study did not collect participant-level token-usage statistics.Source: E. Results · p. 9, Table II · p. 9

Program Comprehension

• All participants reported gaining information, but responses could be overly detailed and their correctness was unreliable, including reported hallucinations.The authors suggest brief-by-default responses and verifiable sources for statements.Source: F. Discussion · p. 9

Overall

• The authors view chat assistants as promising for program comprehension, but responses remain unpredictable in content, length, and correctness.They describe ease of use, flexibility, and relatively low response times as advantages, with response time depending on the LLM.Source: F. Discussion · p. 9

Tool Execution

• Highlighting and color-theme creation received high perceived-correctness and ease-of-use ratings, while tool actions still require verification.The authors conclude chat-assisted execution can reduce perceived effort and improve usability, while recommending stronger tool guardrails and fault handling.Source: F. Discussion · p. 9

Chat-Based Software Restructuring

• Software restructuring received lower, more variable ratings than tool execution, and complex updates need stronger guardrails, clearer previews, and better user controls.Most participants were asked for more information or confirmation; the authors suggest improved prompting may help, while complex tool integration needs special attention.Source: F. Discussion · pp. 9–10

Token Usage (Costs)

• Input-token use was roughly 84 times output-token use, making input reduction the main identified opportunity to lower costs.Suggested options include finer-grained visualization-state access and summarizing or clearing prior chat messages; input tokens usually cost less than output tokens.Source: F. Discussion · p. 10

Participants

• The eleven participants were mostly computer science students, limiting representativeness for professional software developers; the small sample also limits result significance.Some participants had prior ExplorViz or study exposure, and the authors note possible demand characteristics and familiarity with the Spring PetClinic.Source: G. Threats to Validity · p. 10

Task design

• Tasks avoided complex, realistic software-engineering activities requiring deeper work on the software landscape.The landscape and tasks were designed to encourage visualization and assistant interaction without requiring prior ExplorViz knowledge.Source: G. Threats to Validity · p. 10

Task order

• Manual tasks always came before assistant tasks, so fatigue, practice, or carryover knowledge may have influenced responses.Source: G. Threats to Validity · p. 10

Model choice

• Only Claude Sonnet 4.5 was evaluated, and transfer of the findings to other LLMs is unclear.Source: G. Threats to Validity · p. 10

VII. CONCLUSIONS AND FUTURE WORK

• The assistant can request visualization data, such as package hierarchies, and use frontend tools to change the visualization; a modular backend connects it to model providers.Source: §VII · p. 10
• Eleven participants generally found the assistant’s summaries and explanations accurate and its natural-language tool calls easy to use.Source: §VII · p. 10
• Open-ended restructuring showed limitations, leading the authors to call for stronger guardrails, clearer previews, and improved user control.Source: §VII · p. 10
• The assistant’s benefits hinge on grounding in application state and predictable actions, particularly for interactive visualization tasks.Source: §VII · p. 10
• The evaluation is an indication of promise; the authors call for studies across visualization tools, industry comparisons with manual interaction, and longer-term use.Source: §VII · p. 10
• Future implementation work could optimize prompts, instructions, context, and tool calls to improve response quality and reduce token usage.Source: §VII · p. 10

VIII. SUPPLEMENTARY MATERIALS

• A reproduction package includes software artifacts, evaluation results, and a video; the video is also available on YouTube.Source: §VIII · p. 10