A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT

Jules White; Quchen Fu; Sam Hays; Michael Sandborn; Carlos Olea; Henry Gilbert; Ashraf Elnashar; Jesse Spencer-Smith; Douglas C. Schmidt

Source-linked AI summary

A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT

Jules White, Quchen Fu, Sam Hays, Michael Sandborn, Carlos Olea, Henry Gilbert, Ashraf Elnashar, Jesse Spencer-Smith, Douglas C. Schmidt

arXiv:2302.11382v1Feb 21, 2023cs.SEcs.AI

TL;DR

LLMs can produce convincing but factually incorrect text, creating a need for more systematic prompt engineering. This paper develops a catalog and documentation framework for reusable prompt patterns, reporting patterns applied to improve outputs across conversational goals and software-engineering tasks.

Problem
LLMs often generate convincing but factually incorrect text, making reliable output generation an important prompt-engineering challenge.
Method
The paper documents domain-independent prompt patterns in a catalog, explains their composition, and illustrates their use across output and interaction goals.
Results
The catalog presents prompt patterns applied to tasks including visualizations, code artifacts, and fact-checking automation.
Takeaways & Limitations
Prompt patterns offer reusable guidance for systematically engineering conversational LLM outputs and interactions across domains.
Takeaways & Limitations
Meta-language prompts can introduce ambiguity or conflicting semantics and are best used carefully in new conversation sessions.

Abstract

from arXiv · show

Prompt engineering is an increasingly important skill set needed to converse effectively with large language models (LLMs), such as ChatGPT. Prompts are instructions given to an LLM to enforce rules, automate processes, and ensure specific qualities (and quantities) of generated output. Prompts are also a form of programming that can customize the outputs and interactions with an LLM. This paper describes a catalog of prompt engineering techniques presented in pattern form that have been applied to solve common problems when conversing with LLMs. Prompt patterns are a knowledge transfer method analogous to software patterns since they provide reusable solutions to common problems faced in a particular context, i.e., output generation and interaction when working with LLMs. This paper provides the following contributions to research on prompt engineering that apply LLMs to automate software development tasks. First, it provides a framework for documenting patterns for structuring prompts to solve a range of problems so that they can be adapted to different domains. Second, it presents a catalog of patterns that have been applied successfully to improve the outputs of LLM conversations. Third, it explains how prompts can be built from multiple patterns and illustrates prompt patterns that benefit from combination with other prompt patterns.

I. INTRODUCTION … B. Overview of Prompt Patterns

The paper frames prompts as programmable instructions for customizing LLM interactions and outputs, then develops reusable prompt patterns modeled on software patterns to support transferable solutions for software-development tasks. It defines prompt patterns through adapted pattern components that capture intent, rationale, structure, implementation, and consequences.

I. INTRODUCTION: Prompts program LLMs by setting conversational context, rules, important information, and desired output form and content.They can enforce coding styles, flag keywords, and provide related information for software-engineering tasks.
I. INTRODUCTION: Prompt engineering can create new interaction paradigms, including quizzes, simulated terminals, and prompts that suggest additional prompts.The paper presents these capabilities as extending beyond output-type specification or information filtering.
I. INTRODUCTION: The paper introduces domain-independent prompt patterns and a catalog for systematically engineering output and interaction goals across tasks such as visualizations, code artifacts, and fact-checking automation.This catalog is presented as a key contribution for conversational LLM use in software development.
II. COMPARING SOFTWARE PATTERNS WITH PROMPT PATTERNS: The framework documents prompt structures for software tasks in forms that can be adapted across domains, emphasizing reusable knowledge rather than isolated examples or domain-specific prompts.Codifying patterns supports reuse and transfer to similar but nonidentical problems.
A. Overview of Software Patterns: Software patterns provide reusable solutions to recurring problems and document generalized forces, implementation guidance, trade-offs, and examples.Their conventional form includes names, intent, motivation, structure and participants, example code, and consequences.
B. Overview of Prompt Patterns: Prompt patterns adapt the software-pattern concept to LLM output generation, providing codified ways to customize outputs and interactions for specific problems.They can help users and teams enforce constraints, include relevant information, and change interaction formats when automating software development tasks.
B. Overview of Prompt Patterns: Prompt patterns use categories including Output Customization, Error Identification, Prompt Improvement, Interaction, and Context Control.Their pattern names identify the addressed problem, while intent and context describe the problem solved and goals achieved.
B. Overview of Prompt Patterns: The adapted prompt-pattern form records motivation, contextual key ideas, example implementation, and consequences, with guidance for adaptation across contexts.The intent should ideally be domain-independent, although domain-specific patterns can document their applicable context.

C. Evaluating Means for Defining a Prompt Pattern’s Structure and Ideas … A. Summary of the Prompt Pattern Catalog

The paper argues that prompt patterns need an intuitive, idea-level representation rather than formal grammars, and proposes fundamental contextual statements as a reusable framework. It then catalogs prompt patterns for conversational LLM interaction, organizing them by the problems they address and illustrating each with implementation examples.

C. Evaluating Means for Defining a Prompt Pattern’s Structure and Ideas: UML is not necessarily appropriate for documenting prompt patterns because it describes software structures rather than the ideas communicated in prompts.Software patterns commonly use structure and interaction diagrams to define participants and their interactions.
C. Evaluating Means for Defining a Prompt Pattern’s Structure and Ideas: Formal grammars are challenged by the need for approachable communication, varied prompt phrasing, higher-level ideas, and novel symbologies or languages.The passages emphasize that prompts communicate knowledge to diverse users and that ideas may be expressed in ways unknown to a grammar designer.
D. A Way Forward: Fundamental Contextual Statements: Fundamental contextual statements are proposed as simple written descriptions of the important ideas that a prompt should communicate to an LLM.These ideas can be rewritten and expressed in arbitrary ways according to users’ needs and experience.
D. A Way Forward: Fundamental Contextual Statements: The approach is intended to let users understand and adapt statements contextually while expressing the underlying ideas through alternate symbology or wording introduced to the LLM.The benefit comes from capturing the prompt’s underlying ideas rather than fixing one wording.
D. A Way Forward: Fundamental Contextual Statements: The paper aims to support prompt frameworks that can be reused or adapted across LLMs, although its examples were tested with ChatGPT using the ChatGPT+ service.ChatGPT was selected for the examples because of its widespread availability and popularity.
III. A CATALOG OF PROMPT PATTERNS FOR CONVERSATIONAL LLMS: The catalog addresses common problems in conversational LLM interaction and software-task output generation, with each pattern accompanied by implementation samples and examples with and without the prompt.This establishes the catalog as both a classification and an applied collection of prompt-pattern examples.
A. Summary of the Prompt Pattern Catalog: The catalog classifies patterns into Input Semantics, Output Customization, Error Identification, Prompt Improvement, Interaction, and Context Control.Input Semantics includes Meta Language Creation; Output Customization includes Output Automater, Persona, Visualization Generator, Recipe, and Template; Error Identification includes Fact Check List and Reflection; Prompt Improvement includes Question Refinement, Alternative Approaches, Cognitive Verifier, and Refusal Breaker; Interaction includes Flipped Interaction, Game Play, and Infinite Generation; Context Control includes Context Manager.
A. Summary of the Prompt Pattern Catalog: The catalog’s remaining categories target error resolution, input and output improvement, user–LLM interaction, and control of contextual information.Their patterns include fact-checking and reflection, question refinement and alternative approaches, question-asking and game-like or indefinite generation, and context specification.

B. The Meta Language Creation Pattern … 2) Motivation:

The Meta Language Creation Pattern defines custom symbols, words, or statements so an LLM applies their assigned semantics, while emphasizing unambiguous notation and careful session use. The Output Automater Pattern aims to reduce the manual effort, tedium, and errors involved in implementing an LLM’s recommended steps by generating automation artifacts.

2) Motivation:: Meta Language Creation uses fundamental contextual statements to establish meanings that guide the ensuing conversation.
3) Structure and Key Ideas:: The pattern explains symbols, words, or statements through translations or commands that bind shorthand to specified semantics.For example, “X” can mean “Y,” or typing “X” can instruct the LLM to take an action.
4) Example Implementation:: An unambiguous notation can represent graphs by defining nodes and edges with “→” and edge properties with “-[w:2, z:3]→”.The convention maps “a →b” to nodes “a” and “b” connected by an edge, while the extended syntax adds properties such as weights or labels.
5) Consequences:: Ambiguous shorthand can confuse the LLM and degrade its performance or accuracy despite clearly defined language semantics.The paper notes that assigning “a” to Marie Antoinette conflicts with “a” as a commonly used indefinite article.
5) Consequences:: Meta Language Creation is best used carefully in new conversation sessions, with one meta-language per session helping avoid conflicting or unexpected semantics.
1) Intent and Context:: The Output Automater Pattern has the LLM generate a script or other automation artifact that performs the steps it recommends.Its goal is to reduce the manual effort required to implement LLM output recommendations.
2) Motivation:: LLM outputs may require users to modify multiple files manually, making implementation tedious and error-prone.The paper illustrates this with a Python configuration script that suggests files and changes for each file.

3) Structure and Key Ideas: … 3) Structure and Key Ideas:

The section presents the Output Automater and Flipped Interaction patterns as reusable prompt structures that automate multi-step tasks or make the LLM drive goal-focused questioning. Their effectiveness depends on concrete instructions, sufficient context, scoped interaction, and user oversight of generated artifacts.

3) Structure and Key Ideas:: Output Automater prompts specify when multi-step output should trigger automation and require an executable artifact that performs the steps.The scope may be limited to outputs exceeding a user-defined number of steps.
3) Structure and Key Ideas:: A concrete artifact type, such as a Python script, helps the LLM translate general steps into executable automation.The artifact must correspond clearly to automating a sequence of steps.
4) Example Implementation:: The Output Automater can translate instructions into scripts that control computer systems through known scripting interfaces.This creates a pathway for LLMs to integrate quality into and control new computing systems.
5) Consequences:: Output automation requires sufficient conversational context for artifacts to function in their target environment, such as a project’s Mac or Windows file system.It works best when the full application and relevant file-system actions are contained within the conversation.
5) Consequences:: Users should understand generated automation artifacts before execution because LLM inaccuracies make blind acceptance risky.The pattern reduces manual work but does not remove responsibility for understanding the resulting actions.
D. The Flipped Interaction Pattern / 1) Intent and Context:: The Flipped Interaction pattern has the LLM ask questions until it obtains enough information to achieve a specified task or goal.Examples include conducting a quiz or gathering information for a cloud deployment script.
2) Motivation:: By selecting the format, number, and content of questions, the LLM can focus interaction on reaching the goal faster and more accurately.This reverses the usual user-driven conversation and may use knowledge the user initially lacks.
3) Structure and Key Ideas:: Flipped Interaction prompts should state the goal, termination condition, and optionally how many questions to ask per cycle.Specifying the goal focuses questions on relevant outcomes, while scoping duration and question count improves control and usability.

4) Example Implementation: … 5) Consequences:

The examples show how flipped interactions can gather deployment requirements before generating automation, while the Persona pattern directs an LLM to produce outputs associated with a specified role or entity. Both patterns benefit from explicit scope and context, although open-ended or non-human personas can introduce unnecessary questions or contextual assumptions.

4) Example Implementation:: A flipped-interaction prompt asks questions about deploying a Python application to AWS and generates a Python deployment script once sufficient information is collected.The prompt delegates information gathering to the LLM before producing the automation artifact.
4) Example Implementation:: More specific constraints and collected information generally improve outcomes, while menus of AWS services can narrow choices or the LLM can choose unspecified details.The prompt may permit autonomous choices for decisions the user does not explicitly make.
4) Example Implementation:: Open-ended flipped interactions are generic and reusable but may ask additional questions that more context could have avoided.Prompt design must balance how much information the LLM should collect before termination against interaction overhead.
5) Consequences:: Known requirements should be injected into prompts because otherwise the LLM may nondeterministically ask for information or guess an appropriate value.For example, specifying Amazon AWS EC2 avoids leaving the deployment target to multiple interactions.
5) Consequences:: Flipped-interaction prompts should explicitly state the desired level of user knowledge, engagement, and control, from minimal interaction to confirmation of key decisions.These settings determine how much the user participates in and controls the process.
1) Intent and Context:: The Persona pattern gives an LLM a point of view, such as a security expert, to guide which outputs and details it focuses on.Users can request help through a familiar role even when they do not know the exact output details needed.
3) Structure and Key Ideas:: Persona prompts instruct the LLM to act as persona X and provide the outputs that persona would create, with the persona expressed through roles, titles, fictional characters, or historical figures.The output scope can be narrowed, such as asking a teacher for assignments or a security reviewer for code-focused analysis.
5) Consequences:: Personas can represent non-human entities such as Linux terminals or databases, but they may generate contextual assumptions, hallucinated content, or requests for additional context.A simulated terminal can produce imaginary file listings, while a simulated database can request table structure and generate synthetic rows; living or harmful-person personas may be disregarded under privacy and security rules.

F. The Question Refinement Pattern … 2) Motivation:

The Question Refinement pattern helps users formulate more precise questions by suggesting scoped revisions and optionally applying them, while its combinations address narrowing and unfamiliar terminology. The Alternative Approaches pattern exposes users to different ways of solving tasks so they can counter cognitive bias and choose more effectively.

F. The Question Refinement Pattern: The Question Refinement pattern engages the LLM in suggesting better questions, helping users reach accurate answers with fewer interactions.It bridges gaps between the user’s knowledge and the LLM’s understanding.
2) Motivation:: Because users may lack domain expertise, the pattern uses limitations, missing information, and assumptions to improve how questions are phrased.These details can reveal information useful for refining the original question.
3) Structure and Key Ideas:: Within a specified scope, the LLM suggests a better question and may ask whether the user wants to use it instead.The scope prevents indiscriminate rewording, while the optional prompt supports automatic adoption of the refined question.
4) Example Implementation:: For software questions, refinement can incorporate security risks, language or framework details, bugs, modularity, code quality, or reduced external-library dependencies.A Python FastAPI authentication question, for example, can be revised to request framework-specific secure practices.
5) Consequences:: The pattern can make interactions more efficient and accurate, but excessive narrowing may cause users to miss broader information.Combining it with other patterns is presented as one way to address arbitrary narrowing.
5) Consequences:: Combining Question Refinement with Cognitive Verifier generates follow-up questions whose answers support a better revision of the original question.A further combination with Persona can define unfamiliar terms for a specified knowledge level, although LLM-generated refinements may contain inaccuracies.
G. The Alternative Approaches Pattern: The Alternative Approaches pattern ensures that the LLM offers different ways to accomplish a task, encouraging users to evaluate whether their chosen approach best meets their goal.Considering alternatives can also teach users concepts for later follow-up.
2) Motivation:: The pattern addresses cognitive bias by making users aware of alternatives they may not have considered from prior experience.Humans may choose familiar approaches even when those approaches are not best for the problem.

3) Structure and Key Ideas: … 3) Structure and Key Ideas:

The sections describe the Alternative Approaches pattern for constraining, comparing, and selecting viable options, and the Cognitive Verifier pattern for decomposing questions and combining subanswers into a stronger response.

3) Structure and Key Ideas:: It instructs the LLM to list the best alternative ways to accomplish the same task.The instruction can be made more specific or domain-specific to exclude non-viable options.
3) Structure and Key Ideas:: Optional clauses compare each approach’s pros and cons, include the original request, and ask the user which approach to use.These clauses add decision criteria, preserve the initial option, and reduce manual copying.
3) Structure and Key Ideas:: The Alternative Approaches pattern scopes recommendations to a particular goal, topic, or constraint boundary.Examples include implementation decisions and application deployment.
4) Example Implementation:: An implementation applies the pattern to cloud-service deployment by comparing alternatives on cost, availability, and maintenance effort before requesting a selection.The example retains the original deployment approach alongside alternative services.
4) Example Implementation:: The pattern is presented as broadly effective, with refinements including a domain-specific catalog of acceptable alternatives and approved-option incentives.Users can be required to select from approved alternatives while receiving their pros and cons.
1) Intent and Context:: The Cognitive Verifier pattern aims to improve answers by forcing the LLM to subdivide questions into additional questions.The approach addresses both high-level or poorly phrased user questions and evidence that subdivision can improve LLM performance.
2) Motivation:: Its motivation includes users’ unfamiliarity with a domain, laziness in prompt entry, or uncertainty about correct phrasing.These factors can make an initial question too high-level for a concrete answer without follow-up.
3) Structure and Key Ideas:: The pattern generates additional questions to uncover missing context and combines their answers into the final response.The stated goals are a more complete, accurate, and comprehensive answer that incorporates relevant information.

4) Example Implementation: … 5) Consequences:

The examples refine Question Refinement by constraining follow-up questions and adapting terminology to user expertise, while Fact Check List exposes output-dependent facts for verification. These patterns improve accessibility and due diligence but introduce tradeoffs around question scope and output amenability.

4) Example Implementation:: Question Refinement instructs the LLM to generate three additional questions, then combine the answers into a final response to the original question.This fixes the number of follow-up questions rather than leaving it entirely to the LLM.
4) Example Implementation:: The refinement assumes limited user knowledge and requires definitions for terms that are not general knowledge, making follow-up questions more accessible.It guides question creation according to the user’s domain familiarity.
5) Consequences:: Exact question counts constrain how much information users must provide, but may exclude an invaluable N + 1 question or overwhelm users if no limit is imposed.The LLM may instead receive a range or permission to ask additional questions.
2) Motivation:: Fact Check List addresses convincing but factually incorrect LLM outputs by identifying facts or assumptions that materially support the generated statements.Users can perform due diligence on the listed basis of the output.
3) Structure and Key Ideas:: The pattern generates fundamental output facts, inserts them at a specified location, and prioritizes facts whose errors could undermine the output’s veracity.These structural requirements define the core pattern.
3) Structure and Key Ideas:: Placing the fact list after the output lets users understand the statements first and can prompt them to identify additional facts requiring checks.This ordering is preferable when listed facts may contain unfamiliar terms.
4) Example Implementation:: Fact-checking can be scoped to high-risk or unfamiliar topics, reducing cognitive burden; cybersecurity is an example of targeted coverage.The pattern is especially useful when users lack expertise in the output domain, whereas experts may not need it.
5) Consequences:: Fact Check List combines with patterns such as Question Refinement and lets users compare listed facts with the output, but applies only to output types amenable to fact-checking.It can support checking library versions in requirements.txt files, but ChatGPT may refuse to produce a list for code samples.

J. The Template Pattern … 2) Motivation:

The Template Pattern instructs an LLM to place generated content into a precise, application-specific structure, preserving the user’s format when the target format is unknown. It can constrain output effectively, but may filter useful information and hinder combination with other output patterns.

J. The Template Pattern: The pattern ensures that generated output follows a precise template, including application-specific structures such as URLs with content inserted into designated positions.It enables formats the LLM would not ordinarily use for the requested content.
1) Intent and Context:: Application-specific formats must be explicitly described because the LLM may not know the template or where each output component belongs.The format can be conveyed through sample data structures, filled form letters, or similar examples.
3) Structure and Key Ideas:: Users define placeholders, direct the LLM to fit content into them, and require preservation of the supplied formatting and overall template.Placeholder conventions can semantically target insertion locations and indicate which output components should be omitted.
3) Structure and Key Ideas:: The template can usually constrain responses, but extra text before or after it may require experimentation with prompt wording to eliminate.If the LLM already knows the target format, the Template Pattern may be unnecessary.
4) Example Implementation:: Template-based filtering can remove useful descriptions, reasoning, code details, or other information that the LLM might otherwise provide.Users should weigh the benefits of strict formatting against the loss of additional information.
5) Consequences:: The constrained format can make the Template Pattern incompatible with other output-customization patterns, such as combining a URL template with the Recipe Pattern’s step list.The URL template constrains output to a format that does not readily support a list of steps.

3) Structure and Key Ideas: … 4) Example Implementation:

The section presents Infinite Generation as a reusable strategy for producing repeated, rate-limited outputs while incorporating feedback and permitting explicit termination. It also presents Visualization Generator as a pattern that produces tool-compatible visualization inputs, with examples combining templates and automatic tool selection.

3) Structure and Key Ideas:: Infinite Generation repeatedly produces X outputs at a time, allowing reuse of one prompt while limiting output length.Rate limiting is useful when a single output might exceed the LLM’s length limitations.
3) Structure and Key Ideas:: Optional instructions can incorporate user inputs between generations, using feedback to refine output within the original prompt’s context.The additional inputs are leveraged as part of a prescribed prompting strategy.
5) Consequences:: Explicit stop instructions can resolve ambiguity between refinement input and a command to end generation.A stop phrase is useful when user inputs might otherwise be interpreted either as refinements or termination commands.
4) Example Implementation:: The pattern’s sample implementation combines Infinite Generation with a Template to generate repeated URLs containing name and job placeholders.The template preserves formatting while generated text fills the listed placeholders.
L. The Visualization Generator Pattern:: The Visualization Generator pattern uses text generation to create inputs for external tools that produce diagrams or images.Examples include Graphviz Dot and DALL-E, extending LLM outputs with visualizations.
1) Intent and Context:: Because LLMs generally produce text rather than imagery, the pattern converts textual outputs into correctly formatted inputs for visualization tools.This makes results more visually appealing and easier for users to understand.
3) Structure and Key Ideas:: The core visualization instruction asks the LLM to generate X that can be provided to tool Y for visualization.The wording clarifies that the expected output is a tool-consumable image description rather than an image itself.
4) Example Implementation:: An example lets the LLM choose between a Graphviz Dot file and a DALL-E prompt according to the visualization required.Graphviz suits exactly defined graph structures, whereas DALL-E suits realistic or artistic imagery.

5) Consequences: … 2) Motivation:

The Game Play Pattern lets users define topic-focused games through rules while the LLM generates broad content and guides play, potentially extending outputs into visualizations. The Reflection Pattern asks the LLM to explain its rationale, helping users assess validity, clarify assumptions, and debug prompts when answers are incomplete, incorrect, or ambiguous.

1) Intent and Context:: The Game Play Pattern creates a topic-centered game in which the LLM guides gameplay and generates content from user-specified rules.It is particularly effective when rules are limited in scope but the expected game content is broad.
3) Structure and Key Ideas:: Users specify a topic and one or more fundamental game rules, allowing the LLM to determine the game’s content instead of requiring manual creation.More specific topics typically produce more novel and interesting gameplay.
3) Structure and Key Ideas:: Textual games work best when rules fit the LLM’s capabilities, while rich expressive input can support interfaces involving complex scripted actions.The pattern relies on input and output text sequences and can accept scripts describing sequences of actions.
4) Example Implementation:: A cybersecurity game prompts ChatGPT to act as a compromised Linux terminal and respond to typed commands with corresponding terminal output.The example asks the user to investigate how the system was compromised through actions involving processes, files, and network ports.
4) Example Implementation:: The cybersecurity example generated a realistic fake log containing commands that deleted web-server data and replaced it with a backdoor.The displayed command sequence includes rm -rf *, downloading backdoor.php, changing its permissions, and inspecting processes and network sockets.
5) Consequences:: The pattern can combine with Persona, Infinite Generation, and Visualization Generator patterns to support role-based gameplay, continued content generation, and visualized network topology or traffic flows.The target pipeline may include AI generators such as DALL-E, expanding output into the visual domain.
1) Intent and Context:: The Reflection Pattern asks the model to explain the rationale behind answers so users can assess validity and understand how inputs, assumptions, and data shaped outputs.Reflection can clarify confusion, uncover assumptions, and reveal gaps in knowledge or understanding.
2) Motivation:: Reflection helps address incomplete, incorrect, or ambiguous answers and enables users to debug prompts when results fail to meet expectations.It is particularly useful for topics that may be confused with other topics or have nuanced interpretations.

3) Structure and Key Ideas: … 4) Example Implementation:

The catalog’s examples show how prompt patterns can require explanations, assumptions, alternatives, and supporting evidence, while also highlighting risks of inaccessible rationales, inaccurate explanations, and misuse. The Refusal Breaker pattern helps users rephrase unanswered questions, but its use may expose or circumvent guardrails and therefore requires responsible restrictions.

3) Structure and Key Ideas:: The Reflection pattern asks the LLM to explain its reasoning and assumptions after generating an answer, helping users understand and refine their questions.The explanation’s purpose is to help users improve their questions and build trust in responses.
4) Example Implementation:: In software-framework selection, the pattern additionally requests examples or code evidence, justification of the choice, and discussion of ambiguities or limitations.This domain-specific implementation seeks a more complete and accurate response.
5) Consequences:: A limitation is that users unfamiliar with the topic may not understand complex rationales or detect errors and inaccurate assumptions in them.The issue is illustrated by highly technical questions posed by non-technical users.
O. The Refusal Breaker Pattern: The Refusal Breaker pattern aims to help users rephrase questions when an LLM refuses to answer, including because it lacks knowledge or does not understand the wording.It can offer a different question the LLM is better equipped to answer, reducing required user intervention and knowledge about bypassing refusals.
4) Example Implementation:: The pattern can be misused to generate phishing emails or violate policy filters, so organizations and other stakeholders may need to restrict its use and guard against guardrail discovery.Future work could hide the underlying prompt information and rationale from users.
3) Structure and Key Ideas:: Its core instructions activate when a question cannot be answered, ask why, and provide one or more alternative wordings that the LLM could answer.The explanation may reveal assumptions, constraints, or misunderstandings that help the user reword the question.
4) Example Implementation:: The implementation phrases the pattern as: “Whenever you can’t answer a question, explain why and provide one or more alternate wordings of the question that you can’t answer so that I can improve my questions.”A sample response redirects “What is the meaning of life?” toward more answerable questions about philosophical perspectives or common beliefs.

5) Consequences: … 3) Structure and Key Ideas:

The Context Manager Pattern lets users explicitly include, exclude, or reset conversational context to improve relevance, while risking the removal of unseen helpful patterns. The Recipe Pattern organizes partially specified ingredients into a complete sequence of steps toward a stated goal and combines Template, Alternative Approaches, and Reflection patterns.

P. The Context Manager Pattern — 5) Consequences:: The Context Manager Pattern cannot guarantee that users will overcome a refusal or receive an answer to a semantically equivalent variation of the original question.Generated alternate questions may be uninteresting or unhelpful, so the pattern mainly helps determine what the LLM can answer.
P. The Context Manager Pattern — 1) Intent and Context:: The Context Manager Pattern enables users to specify or remove conversational context, giving them greater control over which statements the LLM considers when generating output.Its goal is to focus conversations on specific topics or exclude unrelated topics.
P. The Context Manager Pattern — 2) Motivation:: LLMs may misinterpret the current question or generate irrelevant responses when prior inputs introduce unrelated topics or draw attention to irrelevant statements.Explicit contextual statements can help the LLM better understand the question and generate more accurate responses.
P. The Context Manager Pattern — 3) Structure and Key Ideas:: Context instructions should explicitly list the concepts, facts, instructions, or other statements to include or remove so the LLM can take appropriate action.The passage states that greater explicitness makes the list better, especially when disregarding information discussed earlier.
P. The Context Manager Pattern — 4) Example Implementation:: Users can scope analysis to security aspects, exclude formatting or naming conventions, or reset the conversation with “Ignore everything that we have discussed. Start over.”The “start over” prompt is intended to produce a complete reset of the context.
P. The Context Manager Pattern — 5) Consequences:: A context reset may inadvertently remove helpful patterns that an organization injected into the conversation without the user’s awareness.This can eliminate LLM capabilities while making the loss of functionality unobvious.
Q. The Recipe Pattern — 1) Intent and Context:: The Recipe Pattern constrains the LLM to produce a sequence of steps from partially provided ingredients, combining the Template, Alternative Approaches, and Reflection patterns.Users may know the goal and ingredients but not the precise ordering needed to achieve the outcome.
Q. The Recipe Pattern — 2) Motivation:; 3) Structure and Key Ideas:: The Recipe Pattern states a goal, supplies known intermediate steps, requests a complete sequence, fills missing steps, and identifies unnecessary steps.These statements focus generation on the overall goal, use supplied steps as waypoints or structural constraints, and organize the recipe sequentially.

4) Example Implementation: … V. CONCLUDING REMARKS

The paper illustrates prompt-pattern use through a cloud-deployment recipe and discusses its limitations, relationship to prior work, and concluding lessons. It presents prompt patterns as reusable, composable, and generalizable structures whose catalog remains insufficient as LLM capabilities evolve.

4) Example Implementation:: A cloud-deployment recipe asks the LLM to provide a complete sequence, fill missing steps, and identify unnecessary steps.The example notes that installing dependencies on a virtual machine may be unnecessary when the application is already packaged.
5) Consequences:: The recipe pattern can be biased toward a user’s initially selected steps, even when those steps are unnecessary or the implementation is underspecified.An LLM may incorporate virtual-machine dependency installation instead of flagging it as unneeded.
IV. RELATED WORK: The work complements prior studies by providing a structure for documenting, discussing, and reasoning about prompts to develop mental models for solving common problems.It builds on software-pattern research and prior prompt-design studies spanning image generation, visualization, classification, literature queries, and software debugging.
V. CONCLUDING REMARKS: The framework documents and applies a catalog of prompt patterns analogous to software patterns, offering reusable solutions for users interacting with LLMs across tasks.It provides a structured way to discuss prompting solutions and identify patterns in prompts.
V. CONCLUDING REMARKS: Combining prompt patterns can create larger and more complex capabilities, including cybersecurity games and combinations such as Game Play with Visualization Generator.The examples include fictitious terminal commands stored in an attacker’s .bash history file.
V. CONCLUDING REMARKS: Prompt-pattern catalogs are useful but insufficient, requiring refinement, expansion, new applications, and development into a more expressive pattern language.The authors argue that a pattern language could guide LLM users more effectively.
V. CONCLUDING REMARKS: As LLM capabilities evolve, prompt patterns may become unnecessary, require enhancement, or need replacement, making continued cataloging necessary.Changes in interaction styles and conversation or session management may also affect pattern utility.
V. CONCLUDING REMARKS: Prompt patterns are generalizable beyond software development to domains including story generation, educational games, and topic exploration.The conclusion characterizes these domains as arbitrary applications of the same patterns.