Source-linked AI summary
ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testing
Ian Arawjo, Chelse Swoopes, Priyan Vaithilingam, Martin Wattenberg, Elena Glassman
TL;DR
LLM output evaluation is difficult because users must make sense of many responses, while existing tools often require programming expertise, narrow domains, or closed access. ChainForge addresses this gap with an open-source visual toolkit for comparing models and prompt variations and testing hypotheses. Studies found it useful for real-world investigations and identified exploration, limited evaluation, and iterative refinement as recurring usage modes.
Problem
Evaluating LLM behavior requires making sense of many responses, but tools beyond basic prompting often require programming APIs, narrow-domain expertise, or closed access.
Method
ChainForge is an open-source visual toolkit that supports visual comparison of LLM responses across models and prompt variations with minimal to no coding.
Results
Studies found that users could apply ChainForge to real-world tasks including model selection, prompt iteration, hypothesis testing, and data-processing pipeline prototyping.
Takeaways & Limitations
Prompt engineering and LLM hypothesis testing commonly proceed through opportunistic exploration, limited evaluation, and iterative refinement.
Takeaways & Limitations
The evaluation was qualitative, the in-lab study lasted 75 minutes, and interviewees may have been self-selected users who already found ChainForge useful.
Abstract
from arXiv · showhide
Evaluating outputs of large language models (LLMs) is challenging, requiring making -- and making sense of -- many responses. Yet tools that go beyond basic prompting tend to require knowledge of programming APIs, focus on narrow domains, or are closed-source. We present ChainForge, an open-source visual toolkit for prompt engineering and on-demand hypothesis testing of text generation LLMs. ChainForge provides a graphical interface for comparison of responses across models and prompt variations. Our system was designed to support three tasks: model selection, prompt template design, and hypothesis testing (e.g., auditing). We released ChainForge early in its development and iterated on its design with academics and online users. Through in-lab and interview studies, we find that a range of people could use ChainForge to investigate hypotheses that matter to them, including in real-world settings. We identify three modes of prompt engineering and LLM hypothesis testing: opportunistic exploration, limited evaluation, and iterative refinement.
1 INTRODUCTION
ChainForge is an open-source visual toolkit for comparing LLM outputs and testing prompt- and model-related hypotheses with minimal coding. Studies found it useful for real-world tasks and identified three modes of prompt engineering and hypothesis testing.
- ChainForge supports on-demand hypothesis testing of text-generating LLM behavior on open-domain tasks with minimal to no coding.
- The toolkit targets model selection, prompt template design, and systematic investigation of LLM behavior, including auditing for biases.
- Users applied ChainForge to tasks ranging from plotting models’ understanding of material properties to discovering subtle cross-language biases.
- Across studies, the authors identify opportunistic exploration, limited evaluation, and iterative refinement as three usage modes.
- Many users found ChainForge effective for model selection, prompt iteration, and hypothesis testing, while real-world users also used it to prototype data-processing pipelines.
2 RELATED WORK
LLMOps tools span exploratory prompting and systematic evaluation, but existing options often separate these activities and impose programming, domain, or access barriers. ChainForge is positioned as a visual, open-source toolkit focused on combinatorial comparison and evaluation of LLM responses.
- LLMOps tools address an ecosystem in which LLM users include auditors and developers who are not necessarily ML experts.
- The LLMOps landscape ranges from exploratory playgrounds and chats to systematic evaluation and testing of LLM outputs.
- Few academic prompting systems support systematic evaluation of textual responses, and PromptAid was tested on only one sentiment-analysis task.
- ChainForge borrows visual flow-based concepts while emphasizing exploration, response-quality evaluation, and combinatorial querying across models and prompt variations.
- Existing systematic-testing tools commonly require configuration files, command-line calls, ML engineering knowledge, or programming APIs.
3 DESIGN GOALS AND MOTIVATION
ChainForge was motivated by the need to test evolving, project-specific prompt criteria systematically while preserving rapid experimentation. Its goals combine comparison, evaluation, improvisational iteration, extensibility, and open-source development.
- The authors needed a way to test prompts systematically against project-specific criteria that evolved improvisationally during software development.
- The design targets model selection, prompt template design, systematic evaluation, and improvisational hypothesis testing.
- Prompt template design emphasizes consistency across many possible inputs rather than finding a single good prompt.
- Systematic evaluation requires many parametrized responses, user-defined scoring, navigation, and rapid result skimming through plots.
- The authors also sought an extensible open-source system that handles basic prompting, plotting, and inspection while enabling further research contributions.
4 CHAINFORGE
ChainForge combines visual flow programming with combinatorial prompting, cross-model comparison, evaluation, visualization, and cached iterative workflows. Its examples show how users can test hypotheses, inspect outputs, refine prompts, and share or extend experiments.
- A prompt-injection scenario combines prompt templates, input fields, four models, repeated responses, an evaluator, and a plot to compare robustness.
- In fifteen minutes, GPT-4 appeared most robust, but the team chose less expensive GPT-3.5 after considering the comparison and cost.
- Prompt template chaining lets users compare templates recursively by reusing variables and nesting TextFields nodes at arbitrary depth.
- ChainForge’s combinatorial design queries multiple models across multiple prompt variables, including hierarchically organized variables and metadata.
- The Response Inspector provides grouped-list and table layouts for comparing responses across prompts, variables, and models.
- Figure 5’s ground-truth workflow combines tabular inputs, prompt fields, evaluators, plotted scores, iterative reruns, and query reuse through caching.
- The project gained broad online use and external extensions, including about 5000 local installations, over 1300 GitHub stars, and adaptations by two research teams.
5 EVALUATION RATIONALE, DESIGN, AND CONTEXT
ChainForge was evaluated as an open-ended toolkit through in-lab and interview studies designed to capture both its useful applications and remaining challenges. The studies combined structured prompt-engineering tasks with participant-driven investigations of LLM behavior.
- Evaluation rationale: The evaluation combined demonstrations and user studies to examine which tasks ChainForge supports and which remain challenging.This approach was chosen because controlled usability studies often cover only a narrow subset of toolkit capabilities.
- Research questions: The studies asked about usage patterns, usability and conceptual pain points, useful tasks, and desired tasks outside current features.These questions matched ChainForge’s open-ended design and its focus on personally meaningful investigations.
- In-lab study: The in-lab study used a structured model-and-prompt selection task followed by unstructured exploration of participants’ own hypotheses.Participants first professionalized an email, then added prompts, evaluated response lengths, and investigated self-selected ideas.
- Interview study: The interview study examined eight people who had already used ChainForge for real-world tasks through six semistructured, 60-minute interviews.Interviewees shared screens and walked through something they had created with the toolkit.
- Analysis: Researchers analyzed 32 hours of recordings and interviews using inductive thematic analysis, affinity diagramming, and participant-level behavior tracking.Separate transcript clustering for the in-lab and interview studies was iteratively expanded until in-lab clusters reached saturation.
6 MODES OF PROMPT ENGINEERING AND LLM HYPOTHESIS TESTING
Across studies, participants’ prompt engineering and LLM hypothesis testing followed three suggestive modes: opportunistic exploration, limited evaluation, and iterative refinement. The modes describe changing workflows and mindsets rather than a rigid sequence.
- Overview: Participants generally moved from opportunistic exploration to limited evaluation and then iterative refinement, although some returned to earlier modes.About half of in-lab users, especially those with limited prior experience, never left exploration.
- Opportunistic exploration: Opportunistic exploration uses rapid prompt, input, and hypothesis changes with a few inputs and multi-model comparison.Users prompt, inspect, and revise quick experiments to probe model behavior.
- Limited evaluation: Limited evaluation converts ad-hoc prompting into an at-a-glance evaluation by scoring responses with automated evaluators and visualizing results.Evaluations at this stage are often coarse, such as checking formatting instead of factuality.
- Limited evaluation: Limited evaluation does not require multiple prompts: some participants evaluated a single prompt and only one input value or field.The authors observed indications that participants might have scaled up their tests with more time, while some may have struggled to conceptualize scaling.
- Iterative refinement: Iterative refinement starts with an established pipeline and criteria, then tests parametrization, input complexity, model changes, or prompt edits against a control.Users tweak, test, and refine the pipeline through one-off evaluations.
7 IN-LAB STUDY FINDINGS
In-lab participants used ChainForge to compare models and prompts, interpret multiple representations, and explore open-ended tasks, while revealing subjective judgments and usability barriers. The study also showed that scaling evaluations and learning templating could remain difficult.
- Usability and engagement: 4.19/5.0 was the average interface rating, and 18 participants wanted to use ChainForge again.No participant rated it below three; cited issues were generally minor usability problems.
- Model and prompt choice: Participants weighed trade-offs among criteria and contexts when choosing models and prompts, including explicit requirements and tacit preferences.Different representations surfaced different response aspects and affected theorizing and decision-making.
- Model selection: Eight participants chose PaLM2, seven GPT-4, and six Claude-2 as the better model, with no consensus or consistent reasoning.Participants noticed similar response-style features but valued them differently, including opposite reactions to response length.
- Prompt comparison: Multiple views helped participants compare prompt behavior and revise strategies, such as combining a consistent prompt’s features with another prompt’s textual quality.P17 used response-length plots and tabular views to reason about concision and variance; P4 used a plot to reconsider an earlier choice.
- Open-ended investigations: Participants brought diverse real-world investigations, including auditing model behavior, refining production prompts, and testing concrete criteria with prior data.The reported investigations ranged from understanding program patch files to classifying user attitudes.
- Learning and strategy: Fifteen interview participants said ChainForge affected their AI understanding, while five reported changed prompting or prompt-engineering strategies.Six were surprised by Claude-2 or PaLM2’s performance relative to OpenAI models.
- Challenges and pain-points: Prompt templating created a learning curve, particularly for declaring input variables, and learning appeared related to prior programming expertise.The authors suggest users without programming experience may need extra resources.
- Challenges and pain-points: Some users struggled to scale up or systematize evaluations, including an expert participant whose impressive pipeline ultimately sent only one prompt to multiple models.In-lab researchers were available to help participants implement ideas and overcome conceptual hurdles.
8 INTERVIEWS WITH REAL-WORLD USERS
Real-world users adopted ChainForge for prompt prototyping, model selection, evaluation, and broader LLM data-processing workflows. They valued rapid iteration, combinatorial experimentation, sharing, and open-source extensibility, while encountering hidden features and friction transitioning toward production.
- Use cases and value: All six interface users found ChainForge useful for prototyping and iterating on prompts and pipelines.Their workflows commonly followed a prompt, evaluate, visualize, and revise loop until outputs met desired criteria.
- Use cases and value: Five interviewees used ChainForge to prototype LLM-based data-processing pipelines beyond prompt engineering or model selection.They imported spreadsheet data, applied parameterized prompts, iterated on templates and pipelines, and exported results.
- Use cases and value: Users valued ChainForge’s combinatorial power for exploring multiple prompts, input fields, models, and hypotheses on datasets.One participant contrasted this capability favorably with another tool that could not support comparable multi-field experimentation.
- Pain points: Exporting was the most common pain point when users moved from ChainForge’s strong prototyping stage toward production integration or result sharing.Users requested easier integration with other applications and more shareable reports.
- Pain points: Hidden affordances and opportunistic-exploration friction hindered users, including unfamiliarity with template variables, metavariables, chaining, and direct response inspection.Some users preferred an Inspect Node on the flow for rapid iteration, while others implemented workarounds for features already available.
- Use cases and value: Real-world users extended ChainForge, shared flows and outputs, and relied on source-code access, Inspect Nodes, and export features for collaboration.Consulting and research participants adapted the code for a German LLM provider, GDPR-related needs, and LLM image-model prototyping.
9 DISCUSSION AND CONCLUSION
ChainForge appears useful for model selection, prompt iteration, and hypothesis testing, while real-world use revealed unanticipated data-processing needs. The authors also identify three testing modes and limitations in systematic evaluation and study design.
- Real-world users extended ChainForge and used it for data processing, including creating and sharing LLM-based processing pipelines.
- The authors identify three modes of prompt engineering and LLM hypothesis testing: opportunistic exploration, limited evaluation, and iterative refinement.
- Users weigh model and prompt trade-offs according to criteria, values, preferences, and contexts, making fully automated prompt engineering unsuitable.
- The toolkit better supported implementation and iteration than conceptualization and planning for systematic evaluations.
- The qualitative studies establish a baseline, but controlled quantitative work, longer observation, and broader interview recruitment remain needed.
A.1 Opportunistic exploration mode: Iterating on hypotheses through rapid discovery of model behavior.
In opportunistic exploration, users rapidly compare models and prompt variants to discover unexpected behavior and formulate new hypotheses. P15’s case shows how cross-language comparisons exposed factual, refusal, and source-selection differences.
- P15 compared three models across English and Indonesian prompts and found differing years, percentages, refusals, and statistics.
- Within minutes, the user observed that models differed in facts, language-specific refusals, and facts across languages, while comparisons with BDS showed inaccuracies.
- Inspecting citations suggested that English responses drew on global sources, whereas Indonesian responses cited BPS.
- The user then tested whether specifying organizations would change answers, illustrating hypothesis iteration and cross-model comparison in opportunistic exploration.
A.2 Limited evaluation mode: Setting up an evaluation pipeline to spot-check factual accuracy.
Limited evaluation begins when users move from exploratory inspection to a structured pipeline for extracting and checking outputs. P18 iterated on an LLM-based scorer, scaled inputs, and encountered visualization friction as complexity increased.
- After adding a conductivity-estimation instruction, P18 found GPT-4’s numeric ranges roughly correct for polymer additives.
- P18 moved from prompt inspection to limited evaluation by extracting conductivity values with an LLM Scorer and plotting them in a Vis Node.
- P18 refined the scorer prompt to request numbers without units after observing that the scorer sometimes returned units.
- Adding Polyaniline to the base-polymer variable let P18 scale up the pipeline and quickly compare conductivity scores.
- When input complexity increased, P18 wanted to group visualizations by base polymer but lacked that option.
A.3 Iterative Refinement mode: Tweaking an established prompt and model to attempt an optimization.
Iterative refinement concerns improving an already established prompt and model against explicit criteria. P8 tested alternative models and language variants, then used a simple evaluator to inspect whether outputs contained a forbidden word.
- His criteria required trade-offs: avoiding specific words mattered more than word count, which he perceived Claude as better at controlling.
- P8 entered iterative refinement with a substantially revised prompt for a German e-commerce assistant and ten evaluation criteria.
- He compared GPT-4, Claude, and PaLM2 because he suspected other models or partial English phrasing might improve German outputs.
- P8 abstracted the forbidden-word instruction into a variable and compared German and English versions using a Simple Evaluator.
- The case exemplifies refining an established pipeline to improve outputs according to specific criteria.
B LIST OF NODES
ChainForge’s nodes are grouped by function, with a Comment Node supporting annotations.
- ChainForge organizes its nodes by type and includes a Comment Node for writing comments.The table groups nodes by type and identifies the Comment Node separately.