PaperlayerView exact revision on arXiv

Source-linked AI summary

ChatGPT Prompt Patterns for Improving Code Quality, Refactoring, Requirements Elicitation, and Software Design

Jules White, Sam Hays, Quchen Fu, Jesse Spencer-Smith, Douglas C. Schmidt

arXiv:2303.07839v1cs.SEcs.AI

TL;DR

Specification errors, ambiguity, and omissions may go unnoticed until implementation and deployment. This paper catalogs software-engineering prompt patterns and concludes that LLMs could automate lifecycle tasks through rapid experimentation at multiple abstraction levels.

  • Problem

    Errors, ambiguity, or omissions in software specifications may remain undiscovered until a system is implemented and deployed.

    Source: 5) Consequences · p. 5

  • Method

    The paper develops a software-engineering prompt-pattern catalog spanning lifecycle tasks and examines pattern combinations and sequences.

    Source: §I · pp. 1–2

  • Results

    The authors conclude that LLMs have potential to automate common tasks across the software-engineering lifecycle and enable rapid experimentation at multiple abstraction levels.

    Source: §VII · p. 13

  • Takeaways & Limitations

    Interactive requirements expansion can increase simulated-system fidelity and rapidly generate large requirement sets.

    Source: 5) Consequences · p. 4

    Effective LLM task automation currently requires significant human expertise and scrutiny because ChatGPT can confidently generate incorrect output.

    Source: §VII · p. 13

Abstract

from arXiv · show

This paper presents prompt design techniques for software engineering, in the form of patterns, to solve common problems when using large language models (LLMs), such as ChatGPT to automate common software engineering activities, such as ensuring code is decoupled from third-party libraries and simulating a web application API before it is implemented. This paper provides two contributions to research on using LLMs for software engineering. First, it provides a catalog of patterns for software engineering that classifies patterns according to the types of problems they solve. Second, it explores several prompt patterns that have been applied to improve requirements elicitation, rapid prototyping, code quality, refactoring, and system design.

I. INTRODUCTION · II. PROMPT PATTERN STRUCTURE AND FUNCTIONALITY

The paper presents reusable prompt patterns for software engineering, including constraints intended to improve code quality and a catalog spanning lifecycle activities. It describes a software-pattern-like structure for documenting patterns and shows how scoped instructions and conditional rules guide generated outputs.

I. INTRODUCTION

  • LLMs such as ChatGPT [2] and GitHub Copilot are being adopted for software engineering, and chat interfaces and IDE integrations have made their capabilities easier to access.The paper notes that using these capabilities previously required substantially more time and effort.Source: §I · p. 1
  • In software engineering, prompts are natural-language instructions for generating requirements, code, and other artifacts or simulating systems; they can request outputs or set rules for later interactions.In an IDE, a prompt may take the form of a comment, method definition, or source file.Source: §I · p. 1
  • Reusable prompt patterns [9] encode solutions to recurring interaction problems and can constrain code to separate business logic from side effects and abstract third-party dependencies.These constraints make business logic easier to test and reason about.Source: §I · p. 1
  • The paper extends prior work [9] with a lifecycle-spanning prompt-pattern catalog, from early system simulation to token-limit issues, and examines effective pattern compounds and sequences.The catalog is intended for collaborative use throughout the software life cycle.Source: §I · pp. 1–2

II. PROMPT PATTERN STRUCTURE AND FUNCTIONALITY

  • Each prompt pattern follows a software-pattern-like template: name and classification, intent and context, motivation, structure and key ideas, example implementation, and consequences.Consequences cover pros, cons, and adaptation to different situations.Source: §II · p. 2
  • A typical pattern begins with a conversational scope such as “from now on” and then gives output-generation and prompt-flow rules, which may include conditions for applying specific rules.The scope can also specify a role or a limited number of subsequent prompts.Source: §II · p. 2
  • An adaptation of the Output Automater pattern [9] uses “from now on” to have ChatGPT generate a Python requirements.txt file listing modules used by generated code.This example illustrates a prompt that sets an ongoing rule for future output.Source: §II · p. 2
  • The Output Automator pattern [9] can direct ChatGPT, whenever generated code spans multiple files, to create a Python script that creates or edits those files, reducing manual work and potential errors.The prompt combines a “from now on” scope with a condition and an output rule.Source: §II · p. 2

III. A CATALOG OF PROMPT PATTERNS FOR AUTOMATING SOFTWARE ENGINEERING TASKS … 3) Structure and Key Ideas:

The paper catalogs 13 prompt patterns in four categories for automating software engineering tasks. The Requirements Simulator applies an LLM to system interactions to assess whether requirements capture needed functionality and identify omissions.

III. A CATALOG OF PROMPT PATTERNS FOR AUTOMATING SOFTWARE ENGINEERING TASKS

  • The catalog organizes 13 prompt patterns into four categories to help users navigate and apply them to common software engineering tasks.Table I outlines the initial classifications.Source: §III · p. 2
  • Requirements elicitation and system design address underexplored, consequential areas by helping create and assess requirements and explore concrete specifications, domain-specific languages, and alternative architectures.Simulating system aspects can expose deficiencies earlier, when remediation is less costly and disruptive.Source: §III · pp. 2–3
  • Code Quality patterns promote abstraction and modularity, including interfaces that let developers replace third-party libraries, while Refactoring patterns use high-level descriptions such as pseudocode to guide code changes.The patterns are presented as means to improve both LLM- and human-generated code and to refactor code using LLMs.Source: §III · p. 3

1) Intent and Context:

  • The Requirements Simulator lets stakeholders interactively test whether requirements capture system functionality, with simulation output detailing existing and additional requirements needed for attempted tasks.Its goal is to support elicitation and analysis of requirements completeness.Source: 1) Intent and Context · p. 3

2) Motivation:

  • The simulator addresses late, costly requirement changes by using an LLM to enact user-described tasks and identify requirements missing from the system description.Requirements that inadequately describe system needs are a common source of issues.Source: 2) Motivation · p. 3

3) Structure and Key Ideas:

  • The prompt instructs the LLM to act as the system, follow the requirements, judge whether a requested task is possible and explain why, or specify missing requirements in a designated format.These are the pattern’s core contextual statements.Source: 3) Structure and Key Ideas · p. 3

4) Example Implementation: … 3) Structure and Key Ideas:

The Requirements Simulator uses interactive, requirements-guided exploration to test system capabilities and expose missing requirements as user stories. It can iteratively expand requirements and increase simulation fidelity, while visual wireframes can complement its text-only descriptions.

4) Example Implementation:

  • The task-oriented prompt checks whether a requested action is possible and turns gaps into missing requirements written as user stories.Requirements must already be in the prompt’s current context; they can be entered manually or generated through earlier prompts.Source: 4) Example Implementation · p. 3
  • A screen-by-screen simulator describes the interface and available actions, checks requests against both requirements and the current screen, and updates users when the interface changes.The prompt asks what the user wants to do after describing the current screen.Source: 4) Example Implementation · pp. 3–4

5) Consequences:

  • Interactive exploration expands requirements and increases system-simulation fidelity, enabling rapid generation of large requirement sets.As requirements are added during exploration, the simulation gains fidelity.Source: 5) Consequences · p. 4
  • Because the simulator’s descriptions are text-only, combining it with the Visualization Generator pattern [9] can add screen visualizations and help produce design mockups.The modified prompt requests a DALL-E prompt for generating a wireframe from each textual screen description.Source: 5) Consequences · p. 4
  • Generated wireframes may be easier to reason about than text, spur productive discussion, and inspire ideas through unexpected layouts or interface approaches.A sample wireframe depicts a “My Prompts” screen with prompt titles and descriptions, Delete buttons, and a Back button.Source: 5) Consequences · p. 4, Figure 1 · p. 4

1) Intent and Context:

  • The pattern reviews specifications from non-technical or semi-technical personnel to catch and clarify potential miscommunication or ambiguity early.Source: 1) Intent and Context · p. 4

2) Motivation:

  • The pattern automates some requirements disambiguation before delivery to developers, addressing context that may be lost when managers relay customer-derived requirements.Source: 2) Motivation · p. 4

3) Structure and Key Ideas:

4) Example Implementation: … 2) Motivation:

The examples show how prompt patterns can turn ambiguous requirements into more actionable specifications and generate API designs from natural-language descriptions. Specification Disambiguation is presented as a way to surface weaknesses before development, while API Generator aims to make API design and specification creation faster and earlier.

4) Example Implementation:

  • For a nontechnical product manager’s natural-language requirements, the LLM can sharpen “keep the files secure” into access-control and encryption measures that restrict access and prevent tampering.Follow-up prompts can guide decisions about logged-in users and make requirements more precise and actionable for developers.Source: 4) Example Implementation · pp. 4–5

5) Consequences:

  • Specification errors, ambiguity, and omissions may remain hidden until after deployment, while developers’ assumptions can diverge from designers’ intent; even experienced engineers are prone to incomplete specifications.Source: 5) Consequences · p. 5
  • Specification Disambiguation acts as an automated devil’s advocate to find weak points in requirements, without the social reservations that may keep developers from questioning a senior colleague.Source: 5) Consequences · p. 5
  • For integration across teams, each team can independently identify ambiguities in a shared specification and bring them to a joint meeting as kickoff discussion topics.Source: 5) Consequences · p. 5
  • When specification writers cannot obtain feedback from external audiences, such as consumers of an unreleased API, the pattern can represent those users.The passage gives secrecy and lack of an audience as reasons feedback may be unavailable.Source: 5) Consequences · p. 5
  • Combining Specification Disambiguation with Persona, API Generator, API Simulator, or Requirements Simulator can reveal further ambiguities through different perspectives and transformations into other formats.Source: 5) Consequences · p. 5

1) Intent and Context:

  • API Generator turns natural-language requirements or system descriptions into API specifications, supporting rapid design exploration, earlier formalization, and a starting point for manual refinement.The pattern can generate specifications such as REST APIs.Source: 1) Intent and Context · p. 5

2) Motivation:

  • Reducing the manual cost of API creation can encourage more design exploration, earlier specifications, and careful API design instead of ad hoc integration that relies on code as the source of truth.Source: 2) Motivation · p. 5

3) Structure and Key Ideas: … 4) Example Implementation:

The API Generator turns requirements or system descriptions into API specifications for rapid design exploration, while the API Simulator lets developers interact with a specification before implementation. Together, the patterns support earlier evaluation and refinement of API designs.

4) Example Implementation:

  • The API Generator prompt asks for an OpenAPI specification from listed requirements; more detailed requirements can improve the generated API’s accuracy.The pattern can also be prompted with a simple textual system description, including a cooking-related web application.Source: 4) Example Implementation · pp. 5–6

5) Consequences:

  • The API Generator enables teams to rapidly create and compare multiple API designs, whereas manual specification often limits how many options developers can explore.s22.p1#fd2e32Source: 5) Consequences · p. 6
  • Automating API production can encourage developers to specify APIs earlier, and code-skeleton generation from a specification can accelerate implementation compared with writing it manually.Existing tools can generate specifications from code, but require code to be produced first.Source: 5) Consequences · p. 6
  • The API Generator can be paired with the API Simulator to assess API ergonomics and task difficulty, and with Data-guided Refactoring to revise the API.Source: 5) Consequences · p. 6

1) Intent and Context:

  • The API Simulator lets developers interact with an API from its specification, using synthetic data, sample requests, usage guidance, and error explanations.It can help uncover issues, omissions, and awkward designs through early interaction.Source: 1) Intent and Context · p. 6
  • Existing API simulation tools require setup and may offer limited synthetic data or strict HTTP/code interaction, while early API-design interaction can reveal design problems.Source: 1) Intent and Context · p. 6

3) Structure and Key Ideas:

  • For API simulation, the LLM acts as a system described by specification X, receives requests in format Y, and responds in format Z.Source: 3) Structure and Key Ideas · p. 6

4) Example Implementation:

  • A simulator can use an OpenAPI specification to accept plain-text HTTP requests and return corresponding HTTP responses; input may instead describe user actions or state changes.The specification may be OpenAPI or another programmatic interface or API specification language.Source: 4) Example Implementation · p. 6

5) Consequences: … 4) Example Implementation:

The Consequences section describes how natural-language customization of simulated APIs supports exploration and reusable examples. The Few-shot Code Example Generation pattern uses generated usage examples to convey system design within prompt limits.

5) Consequences:

  • Natural-language customization lets users vary simulated API scenarios and interaction strictness, facilitating rapid API exploration.Users can specify assumptions such as valid OAuth tokens or sample users, and choose strict HTTP-request validation or correction of pseudo-data formatting.Source: 5) Consequences · pp. 6–7
  • Generated API usage examples can become few-shot examples or help users assess task difficulty and the effort required to accommodate later changes.The paper identifies Change Request Simulation as a pattern for reasoning about effort under changing assumptions across the software life-cycle.Source: 5) Consequences · p. 7

1) Intent and Context:

  • The pattern generates usage examples that can stand in for code in later prompts, conveying code function more token-efficiently and reminding the LLM of prior design.Few-shot learning supplies limited example data in a prompt; examples of proper code usage may remind the LLM of a system’s design from earlier conversations.Source: 1) Intent and Context · p. 7

2) Motivation:

  • Because large systems can exceed LLM token limits and manual few-shot example creation may be infeasible, usage examples can communicate design and proper use.Examples may cover code, APIs, state transitions, or other specification usage.Source: 2) Motivation · p. 7

3) Structure and Key Ideas:

  • The pattern asks the LLM to create N system-usage examples, make their coverage as complete as possible, and optionally base or focus them on public interfaces.The prompt begins by saying that system X will be provided.Source: 3) Structure and Key Ideas · p. 7

4) Example Implementation:

  • 10 usage examples for new-user registration from an OpenAPI specification can remind later prompts of API design and convey it more concisely than equivalent natural-language statements.The example implementation requests examples focused on creating new users.Source: 4) Example Implementation · p. 7
  • Code examples can compactly convey operation ordering and required data, while the implementation requests complete usage examples based on the code’s public interfaces.The prompt asks for 10 examples demonstrating use of the provided code.Source: 4) Example Implementation · p. 7

5) Consequences: … 2) Motivation:

The patterns help developers explore system use and architectural alternatives, while DSL Creation lets an LLM produce reusable, token-conscious languages for describing system concepts. Their benefits depend on choosing suitable examples and syntax, since compact formats can be harder to interpret and may require upfront specification.

5) Consequences:

  • Example usage can capture expected system use early and later explain past design decisions to the LLM; paired with an API Simulator, it enables interaction recording and generated examples.The pattern can be used early in the design cycle and supplemented with additional generated examples.Source: 5) Consequences · p. 7
  • Example usage is useful when it conveys constraints, assumptions, or expectations more compactly than prose, although a document such as OpenAPI may use fewer tokens.The passage notes that example usage has been effective for LLM problem solving [17], even when it is not the most token-efficient mechanism.Source: 5) Consequences · p. 7

F. The Domain-Specific Language (DSL) Creation Pattern — 1) Intent and Context: / 3) Structure and Key Ideas:

  • DSL Creation lets an LLM design and explain a domain-specific language, then store generated explanations and examples to reintroduce the DSL in future prompts.The generated examples can serve as few-shot examples.Source: 1) Intent and Context · p. 7

F. The Domain-Specific Language (DSL) Creation Pattern — 1) Intent and Context:

  • DSLs can express system aspects more succinctly and token-efficiently than other formats, but their syntax and semantics must first be explained to the LLM.Token-efficient inputs matter when large software projects require more context than a prompt can hold; creating a DSL can be time-consuming.Source: 1) Intent and Context · p. 8

F. The Domain-Specific Language (DSL) Creation Pattern — 5) Consequences:

  • DSL Creation may facilitate design without hitting token limits, but syntax rules affect space efficiency, and a text format’s concept density may not yield token efficiency.The passage contrasts XML-based syntax, which is likely more space-consuming, with YAML-based syntax.Source: 5) Consequences · p. 8
  • Conventions and implicit syntax rules can improve DSL token efficiency, but may make the language harder for unfamiliar users to interpret.Few-shot examples can teach DSL use and help the LLM self-document it for later prompting.Source: 5) Consequences · p. 8

G. The Architectural Possibilities Pattern — 1) Intent and Context:

  • The Architectural Possibilities pattern generates alternatives for developers to consider and supports interactive refinement through added constraints or different architectural views.Views may include file layout, modules, services, communication patterns, and infrastructure.Source: 1) Intent and Context · p. 8

G. The Architectural Possibilities Pattern — 2) Motivation:

  • Architecture design can demand substantial cognitive effort, limiting the alternatives developers consider, including architectures they may not know; exploring more alternatives is therefore important.The motivation is especially relevant when mapping architectures to system requirements.Source: 2) Motivation · p. 8

3) Structure and Key Ideas: … 5) Consequences:

The patterns support architecture exploration by focusing prompts on a specified system aspect and support change-impact analysis at levels ranging from modules to code. Their usefulness depends on providing concrete context and can be extended through rapid implementation, simulation, and iterative refinement.

4) Example Implementation:

  • A “with respect to” prompt clause focuses architectural descriptions on a chosen aspect, such as modules, REST APIs, services, storage, or deployment.The example asks for three architectures and specifies that each should describe modules and their functionality.Source: 4) Example Implementation · p. 9

5) Consequences:

  • For performance-sensitive applications, the Architectural Possibilities pattern can generate implementations and workload tests to compare architectures and cloud costs.Examples include testing a monolithic application against microservices under varied workloads.Source: 5) Consequences · p. 9

1) Intent and Context:

  • Change Request Simulation helps stakeholders reason about a proposed change’s complexity and impact, including which modules may need modification.It can support discussion of requirements changes and serve as a potentially unbiased estimator of scope and impact.Source: 1) Intent and Context · p. 9

2) Motivation:

  • Rapid feedback on change impacts can help stakeholders discuss schedule, cost, and other risks; distrust may make an unbiased outside opinion useful..Source: 2) Motivation · p. 9

4) Example Implementation:

  • A change simulation can use an earlier OpenAPI specification to identify affected functions and files, or identify lines to change when the relevant code fits in the prompt.The resulting list can help estimate change cost from file and function complexity and counts.Source: 4) Example Implementation · p. 9

5) Consequences:

  • Change-impact estimates require sufficient system context; using patterns such as API Generator to seed the conversation and specifying a concrete change can improve the analysis..Source: 5) Consequences · p. 9
  • LLM token limits constrain detailed analysis of sweeping changes, while smaller changes may be analyzed precisely in terms of files, functions, or lines.Broad changes generally require reasoning at a higher level of abstraction.Source: 5) Consequences · p. 9
  • To gain detail within token limits, iteratively identify affected features or modules, then refine the prompt around each until sufficient detail is available.Repeat the process for high-level modules to estimate the overall impact.Source: 5) Consequences · pp. 9–10

V. CODE QUALITY & REFACTORING PATTERNS … 4) Example Implementation:

The Code Clustering Pattern gives LLMs property-based guidance for separating and grouping code, helping generate higher-quality code and refactor existing code. It can specify desired properties and examples so the LLM can restructure code accordingly.

V. CODE QUALITY & REFACTORING PATTERNS

1) Intent and Context:

  • It separates code into functions, classes, or other groups based on properties such as purity, business logic, or HTTP handling.The pattern can be used to guide generated code, refactor human-produced code, or review code.Source: 1) Intent and Context · p. 10

2) Motivation:

  • Because code organization affects changeability, extensibility, and maintainability, missing clustering guidance can lead LLMs to produce monolithic, brittle, or messy code.The pattern supplies the clustering context needed to guide code generation toward higher quality.Source: 2) Motivation · p. 10

3) Structure and Key Ideas:

4) Example Implementation:

  • A sample prompt separates functions with side effects, such as file, database, or network access, from functions without side effects.One property can be defined as the converse of the other; opposite properties are a common implementation approach.Source: 4) Example Implementation · p. 10
  • Effective clustering properties include side effects, architectural tiers, and cohesive features grouped into separate files or file groups.Separating side effects can support testing and isolated reasoning about business logic.Source: 4) Example Implementation · p. 10
  • Properties that are well understood by the LLM may need less prompt design, while custom properties can be conveyed through natural-language descriptions and few-shot examples.The pattern can combine with the Few-shot Code Example Generator pattern to create clustering examples for in-context learning.Source: 4) Example Implementation · p. 10

5) Consequences: … 3) Structure and Key Ideas:

The patterns show how explicit prompts can improve code structure: separate business logic from third-party libraries and invoke established design principles such as SOLID. Their benefits depend on providing the LLM with the needed structural guidance, and suitable abstractions may require examples of alternative libraries.

5) Consequences:

  • An LLM’s code may solve the stated problem while missing unrequested structural goals, so perceived quality depends on supplying the relevant context in the prompt.Implicit requirements, such as illustrating particular clustering properties, are not known to the LLM unless provided.Source: 5) Consequences · p. 10

B. The Intermediate Abstraction Pattern — 2) Motivation:

  • LLMs often generate procedural code with weak abstraction and modularity, and repeated feature additions can produce long functions that are harder to maintain.The Intermediate Abstraction Pattern addresses this by isolating cohesive concepts so edits can be scoped to a function or class.Source: 2) Motivation · p. 10, 1) Intent and Context · p. 10

B. The Intermediate Abstraction Pattern and C. The Principled Code Pattern — 3) Structure and Key Ideas:

  • The patterns’ structure is introduced through fundamental contextual statements.The supplied passages introduce these statements but do not include their contents.Source: 3) Structure and Key Ideas · pp. 10–11

B. The Intermediate Abstraction Pattern — 4) Example Implementation:

  • The Intermediate Abstraction Pattern asks the LLM to separate business logic from third-party libraries and insert an intermediary abstraction where the logic uses a library.This can reduce exposure to dependency risks such as breaking changes that complicate integration and restrict access to future security updates.Source: 4) Example Implementation · p. 11

B. The Intermediate Abstraction Pattern — 5) Consequences:

  • A good abstraction may be difficult to design from one library alone because dependencies can have fundamentally different architectures and interfaces.The paper suggests using Few-shot Example Generator to create examples of comparable libraries, then asking the LLM to refactor the interface for alternatives.Source: 5) Consequences · p. 11

C. The Principled Code Pattern — 2) Motivation:

  • Developers may struggle to specify all the natural-language rules behind good design practices, even when they know the name of a design methodology.Long articles and papers explain how good design practices apply across languages and frameworks.Source: 2) Motivation · p. 11

4) Example Implementation: … 4) Example Implementation:

The section illustrates prompt patterns that apply named design principles and expose code assumptions, helping developers guide refactoring and understand potential constraints. Their usefulness depends on whether the LLM has learned examples of the named concepts.

4) Example Implementation:

  • The SOLID example asks the LLM to apply five named design principles: single responsibility, open-closed, Liskov substitution, interface segregation, and dependency inversion.The prompt says to follow SOLID whenever writing, refactoring, or reviewing code.Source: 4) Example Implementation · p. 11

5) Consequences:

  • The named-principle pattern works best when training material explains its application across codebases; less mainstream languages such as Prolog may have fewer examples.The paper notes that more widely known design principles are likely to have more examples in the LLM's training data.Source: 5) Consequences · p. 11
  • Like the Persona Pattern [9], this approach depends on names known before training; in-context learning and few-shot examples may illustrate newer styles.Coding or design styles that emerged after the training date are not accessible through the named-description approach alone.Source: 5) Consequences · p. 11

D. The Hidden Assumptions Pattern — 1) Intent and Context:

  • The Hidden Assumptions Pattern asks the LLM to identify and describe code assumptions, helping users surface forgotten assumptions and account for them in decisions.The pattern is intended both to identify assumptions and to remind users of assumptions they may have overlooked.Source: 1) Intent and Context · p. 11

D. The Hidden Assumptions Pattern — 2) Motivation:

  • Because any code can contain hidden assumptions, users who miss them may misuse or incorrectly modify it; this risk is greater for LLM-generated code they know less well.The paper notes that users may be less familiar with code produced for them by an LLM.Source: 2) Motivation · pp. 11–12

D. The Hidden Assumptions Pattern — 3) Structure and Key Ideas:

  • The pattern scopes the code, lists its assumptions, and optionally estimates how difficult each assumption would be to change or how likely it is to change.These are the pattern's fundamental contextual statements.Source: 3) Structure and Key Ideas · p. 12

D. The Hidden Assumptions Pattern — 4) Example Implementation:

  • One example asks which assumptions are hard to change, enabling developers to identify future-change liabilities and request refactoring when a difficult-to-change assumption is expected to change.The sample prompt asks how hard each assumption would be to change given the current code structure.Source: 4) Example Implementation · p. 12
  • A second example identifies assumptions that tightly couple code to MongoDB, so the list can guide refactoring toward changing to MySQL.The sample prompt specifically asks which assumptions make the database change difficult; the list can then be used to request refactoring that eliminates them.Source: 4) Example Implementation · p. 12

5) Consequences: … 1) Intent and Context:

The Pseudo-code Refactoring Pattern gives users control over code structure without requiring detailed specifications, while the Data-guided Refactoring Pattern lets them request changes by supplying a new data-format schema. The passages also note that refactoring may exceed the outline or miss assumptions when the provided context is incomplete.

5) Consequences:

  • The earlier pattern may miss assumptions outside the supplied code context, so its suggestions should not be treated as a complete source of truth.Source: 5) Consequences · p. 12

E. The Pseudo-code Refactoring Pattern — 1) Intent and Context:

  • Pseudo-code lets users guide an algorithm, flow, or other code aspects without specifying every detail, while expecting correct, runnable output.The LLM adapts generated or refactored code to the pseudo-code template.Source: 1) Intent and Context · p. 12
  • Pseudo-code offers a middle ground between unrestricted refactoring and typing exact code structures, which may duplicate the LLM’s work and reduce its benefit.Source: 1) Intent and Context · p. 12

E. The Pseudo-code Refactoring Pattern — 4) Example Implementation:

  • The example uses an outline to refactor a larger code body without specifying task details or line-level traceability, leaving the LLM to infer intent and map functionality.Source: 4) Example Implementation · p. 12

F. The Data-guided Refactoring Pattern — 1) Intent and Context:

  • The Data-guided Refactoring Pattern asks the LLM to determine changes needed to use a new data format from its schema, automating this common refactoring task.Source: 1) Intent and Context · pp. 12–13

3) Structure and Key Ideas: … VII. CONCLUDING REMARKS

The sections describe how prompt patterns can reduce refactoring effort, complement existing software-engineering and prompt-engineering work, and help LLMs support software tasks. They also emphasize that effective use still requires human expertise and further work on output quality assurance and versioning.

4) Example Implementation:

  • The example asks the LLM to refactor an execute-graph function into a graph-and-sorted-nodes format, using ellipses so the LLM can fill in concrete details.The graph format’s use is unspecified and could be input, output, or internal to the function.Source: 4) Example Implementation · p. 13

5) Consequences:

  • The refactoring pattern can automate or bootstrap changes required by new data formats, reducing manual effort and potentially lowering costs and speeding system development.Data-format changes can have large-scale effects on a system.Source: 5) Consequences · p. 13

VI. RELATED WORK

  • Prompt design patterns complement software design patterns,, [20], by focusing on LLM interactions for producing and maintaining software-reliant systems rather than on system design itself.Source: §VI · p. 13
  • This work complements prompt-engineering and tool-use studies, [23], [17] with interaction patterns for specific software-engineering problems.Source: §VI · p. 13
  • Related work includes security comparisons of LLM and human code generation [5], code-quality studies [12], [23]–, and bug-fixing interaction patterns [26], [27]; this paper documents patterns to improve code quality and reduce errors.The authors note that future prompt patterns can be quantitatively compared for effectiveness in addressing code-quality issues.Source: §VI · p. 13

VII. CONCLUDING REMARKS

  • Prompt patterns can reduce errors and expose LLM capabilities such as simulating systems from requirements, generating API specifications, and identifying code assumptions.These capabilities are described as difficult to automate with existing technologies.Source: §VII · p. 13
  • LLMs could accelerate software engineering through automation and rapid experimentation across abstraction levels, if effective prompts and guidance are cataloged for lifecycle use.The paper identifies codifying a prompt catalog and guidance for combining prompts at different lifecycle stages as key to leveraging these capabilities.Source: §VII · p. 13
  • Human expertise and close scrutiny remain necessary because ChatGPT can confidently produce incorrect output; prompt patterns mitigate some issues, but quality assurance and versioning need further work.The paper calls for further work to ensure LLM output is accurate and useful in practice.Source: §VII · p. 13

Related papers

Loading 2303.07839v1…