PaperlayerView exact revision on arXiv

Source-linked AI summary

What is it like to program with artificial intelligence?

Advait Sarkar, Andrew D. Gordon, Carina Negreanu, Christian Poelitz, Sruti Srinivasa Ragavan, Ben Zorn

arXiv:2208.06213v2cs.HCcs.AIcs.PL

TL;DR

LLM-assisted programming shares some properties with established forms of programmer assistance, but its technical possibilities and practical experience differ. The paper examines developer reports and a spreadsheet study, finding a distinct programming experience and challenges for non-expert users.

  • Problem

    How LLM-assisted programming relates to established forms of programmer assistance, and how non-expert users handle generated code, remain important questions for understanding this distinct programming experience.

    Source: §9.1 · p. 18, §10 · p. 19

  • Method

    The paper examines developer experience reports and studies 20 data analysts using natural-language formula generation in the GridBook spreadsheet system.

    Source: §7.1 · p. 10, §7.2 · p. 12, §7.3 · p. 13

  • Results

    LLM assistance shares some properties with compilers, pair programming, and search/reuse but has a distinct programming experience, while spreadsheet use surfaces intent, correctness, and comprehension challenges.

    Source: §9.1 · p. 18, §9.3 · p. 18, §10 · p. 19

  • Takeaways & Limitations

    Non-expert spreadsheet users may struggle to specify computational intent and understand generated formulas, complicating evaluation, customization, debugging, and reuse.

    Source: §9.1 · p. 18, §9.3 · p. 18

    For novices or users unfamiliar with the target language, generated code can be exceedingly difficult to understand; Python’s richer data-wrangling libraries also make Excel formulas less likely to solve some user problems.

    Source: §9.5 · p. 19

Abstract

from arXiv · show

Large language models, such as OpenAI's codex and Deepmind's AlphaCode, can generate code to solve a variety of problems expressed in natural language. This technology has already been commercialised in at least one widely-used programming editor extension: GitHub Copilot. In this paper, we explore how programming with large language models (LLM-assisted programming) is similar to, and differs from, prior conceptualisations of programmer assistance. We draw upon publicly available experience reports of LLM-assisted programming, as well as prior usability and design studies. We find that while LLM-assisted programming shares some properties of compilation, pair programming, and programming via search and reuse, there are fundamental differences both in the technical possibilities as well as the practical experience. Thus, LLM-assisted programming ought to be viewed as a new way of programming with its own distinct properties and challenges. Finally, we draw upon observations from a user study in which non-expert end user programmers use LLM-assisted tools for solving data tasks in spreadsheets. We discuss the issues that might arise, and open research challenges, in applying large language models to end-user programming, particularly with users who have little or no programming expertise.

1. Introduction

LLM-assisted programming resembles familiar forms of assistance but has distinct properties and challenges; applying it to non-expert spreadsheet programming raises additional issues.

1. Introduction

  • Large language models mark a qualitative and quantitative step-change in code and natural-language generation, following cumulative innovations in embeddings, transformers, large corpora, and pretraining.Source: §1 · p. 1
  • LLM tools act like advanced autocomplete, generating multiple lines from natural-language prompts, code, or mixtures, including function bodies, tests, and repetitive patterns.Source: §1 · pp. 1–2
  • Lab and telemetric studies report that developers appreciate these tools despite no strong effects on task time or correctness, with prompt framing and code checking remaining challenges.Source: §1 · p. 2
  • Longitudinal reports echo prompt-framing and debugging challenges, while many developers report that the tools reduce task time.Source: §1 · p. 2
  • Search, compilation, and pair programming share some properties with LLM-assisted programming but are inadequate metaphors for its distinct blend of properties.The paper frames it as a new way of programming with its own challenges.Source: §1 · p. 2
  • A spreadsheet study identifies challenges in intent specification, correctness, comprehension, LLM tuning, and end-user behaviour.The authors call for further study of LLM-assisted end-user programming.Source: §1 · p. 2

2. Prior conceptualisations of intelligent assistance for programmers

Prior assistance ranges from direct manipulation and programming by demonstration to languages, synthesis, and intelligent tools supporting multiple programming activities. These approaches distribute inferential work differently between people and systems.

2. Prior conceptualisations of intelligent assistance for programmers

  • Direct manipulation works for limited tasks when the interface closely represents the artifact, but programs describe multiple possible future executions that are difficult to render in the present.Source: §2 · p. 2
  • Programming by demonstration lets users demonstrate desired behaviour for generalization into executable programs, but it has not achieved widespread success as an end-user programming tool.Its idea persists in macro recording and is seeing renewed interest alongside robotic process automation.Source: §2 · pp. 2–3
  • Compilers translate high-level languages into machine code, while Prolog assigns the programmer the task of declaring what to compute and leaves how to compute it to the interpreter.Source: §2 · p. 3
  • Program synthesis can transform specifications into executable code, while programming by example derives code from input-output examples, as in Excel Flash Fill.Source: §2 · p. 3
  • Intelligent tools support authoring, transcription, and modification, as well as code navigation, debugging, selective undo, and learning.Source: §2 · p. 3
  • Language models applied to code exploit lexical patterns that traditional compilers and verifiers do not use, supporting tools such as code completion, debugging, analysis, and synthesis.The naturalness hypothesis holds that software corpora share statistical properties with natural-language corpora that can be exploited for better software tools.Source: §2 · p. 3

3. A brief overview of large language models for code generation

Large pre-trained language models use transformer-based architectures and prompting to generate code from natural-language and code inputs. Benchmarks show substantial but imperfect code-generation performance, with reliability and consistency concerns.

3.1. The transformer architecture and big datasets enable large pre-trained models

  • Transformers combine word embeddings, self-attention, and positional encodings; pre-trained models commonly learn from next-word prediction or sequence-gap filling.Source: §3.1 · pp. 3–4
  • Pre-trained models can be adapted through fine-tuning or prompted to predict directly for a downstream task, an approach described as Pre-train, Prompt, Predict.Source: §3.1 · p. 4

3.2. Language models tuned for source code generation

  • Code generation takes natural-language comments and code snippets as input and produces code, with public code repositories providing large training corpora.Figure 2 illustrates pattern completion from two examples and function-body generation from a signature and first line.Source: §3.2 · p. 4, Figure 2 · p. 5
  • Codex solves an average of 47/164 problems in one attempt on the HumanEval code-generation benchmark.HumanEval comprises 164 hand-written programming problems.Source: §3.2 · p. 4
  • On 10,000 programming problems, Codex solves around 3% within five attempts, versus AlphaCode’s 4–7%; AlphaCode achieves a 34% success rate on CodeContests.Source: §3.2 · p. 4
  • Generated code may copy training solutions, contain errors or undefined references, and vary across runs even under identical settings.The paper notes that correctness is not guaranteed and errors can propagate through generations.Source: §3.2 · pp. 4–5

4. Commercial programming tools that use large language models

Commercial tools built on large language models bring code generation into programming editors, principally as authoring assistance. Their interfaces and capabilities vary, with Copilot offering broader generation than more traditional autocomplete tools.

4. Commercial programming tools that use large language models

  • Codex is GPT fine-tuned on publicly available source code and is commercialized as an API that other applications can use.Source: §4 · p. 6
  • GitHub Copilot uses Codex and project files or repository paths as context to generate code at the editor cursor like autocomplete.Source: §4 · p. 6
  • Suggested Copilot uses include completing function bodies from comments and signatures, writing boilerplate, and defining tests; accepted completions remain editable.Source: §4 · p. 6
  • Commercial LLM assistance has primarily targeted authoring, while IntelliCode and Tabnine are described as closer to traditional autocomplete and narrower in scope than Copilot.CodeWhisperer was announced as a similar feature, but was not widely available at the time described.Source: §4 · pp. 6–7

5. Reliability, safety, and security implications of code-generating AI models

Code-generating AI raises reliability, safety, and security concerns that current functional-correctness benchmarks do not fully capture. The paper also highlights over-reliance and the possibility that public training data can contribute to insecure suggestions.

5. Reliability, safety, and security implications of code-generating AI models

  • Functional-correctness benchmarks based on unit tests do not assess code readability, completeness, or the potential errors developers must address.Evaluating complex software artifacts requires more than checking whether whole functions pass tests.Source: §5 · p. 7
  • Over-reliance can lead users to assume generated code is correct and secure, but its consequences for software quality remain an open question.The paper notes that deployed tools advise users to review and test generated code as they would code from an external source.Source: §5 · p. 7
  • Public-repository training data may include low-quality or vulnerable code; one early Copilot study found insecure suggestions in some tested scenarios.Possible mitigations include screening generated code with existing security tools, improving training-data quality, and fine-tuning to reduce known insecure patterns.Source: §5 · p. 7

6. Usability and design studies of AI-assisted programming

Prior usability studies report both perceived benefits and evaluation burdens in AI-assisted programming. Findings vary by task and interaction: programmers use tools for speed, exploration, and retrieval, but still face challenges specifying intent and assessing suggestions.

6. Usability and design studies of AI-assisted programming

  • 19 of 24 participants preferred Copilot, and 23 of 24 perceived it as more helpful, despite more task failures and no significant task-time effect.Assessing generated-code correctness was difficult and could become an efficiency bottleneck when a fundamental flaw led to unsuccessful repair or debugging.Source: §6 · p. 7
  • Acceptance rate—the ratio of accepted to shown suggestions—was the strongest predictor of users’ perceived productivity in a survey of 2,047 Copilot users.Aggregate acceptance followed daily and weekly rhythms, but each user was less likely to accept suggestions outside their normal working hours; replication was identified as future work.Source: §6 · p. 8
  • Simple queries were more likely to be answered correctly through generation, while complex, multi-step queries favored retrieval; the plugin had no significant effect on time or correctness.Retrieved snippets were edited more often, and participants valued the tool for finding snippets they knew but could not recall.Source: §6 · p. 8
  • 60% of natural-language queries in one study were too underspecified for a human expert to implement, and participants needed experimentation to formulate queries.Participants reported mostly neutral or somewhat positive views (30 of 31); the interaction worked better when they already knew the target APIs and frameworks.Source: §6 · p. 8
  • In a week-long study, novice programmers mainly used natural-language queries, while experts more often mixed code into requests; participants struggled to understand the model’s required query “syntax.”The authors describe this difficulty as an “uncanny valley” and suggest prompt rewording, task breakdowns, and better onboarding as possible responses.Source: §6 · p. 8
  • Copilot use involved acceleration for well-formed intents and exploration for planning; long suggestions could interrupt flow during acceleration.In exploration, participants more often used natural-language comments, which they sometimes cleaned after accepting a suggestion.Source: §6 · p. 9
  • In a study of 21 participants, Copilot-generated code was comparable in complexity and readability to human-authored code.Source: §6 · p. 9
  • Bing Developer Assistant saved an average of 28% of task time versus regular web search in a study of 45 short programming tasks.The figure illustrates identifier-aware retrieval: the “@line” query hint is used to rewrite a retrieved Stack Overflow snippet.Source: §6 · p. 9, Figure 5 · p. 9

7. Experience reports

The experience reports draw on developers’ blog posts and online discussions to examine long-term use of LLM programming tools. They describe both useful code-generation workflows and challenges around prompting, checking, and debugging.

7. Experience reports

  • The reports are drawn from developers’ blogs and online community discussions, which the authors use as qualitative evidence about longer-term experience.Exploratory programming, where intent is not yet defined, falls outside the framework discussed here.Source: §7 · pp. 9–10, §7.1 · p. 11

7.2. The activity of programming shifts towards checking and unfamiliar debugging

  • Developers also report technically correct suggestions that use poor practices, increased code verbosity, and concerns about private information and copyright.The paper notes that it does not examine privacy and copyright concerns in depth.Source: §7.1 · p. 10, §7.2 · p. 13

7.1. Writing effective prompts is hard

  • Developers report that these tools can reveal unfamiliar APIs and coding conventions, including structured Go error handling and consistent SQL foreign-key names.One developer also reported discovering higher-level or better-fitting API methods.Source: §7.1 · pp. 10–11
  • Prompting is a skill: developers adjust detail and break tasks into steps, while LLMs can produce usable results across a wide range of abstraction levels.This flexibility lowers the floor for getting usable results but raises the ceiling for reaching maximum productivity; specific prompts can still fail when APIs have changed, as with fastai.Source: §7.1 · pp. 10–11

7.2. The activity of programming shifts towards checking and unfamiliar debugging

  • Generated code can shift effort from writing to checking, while plausible, subtle bugs make familiar debugging intuitions less reliable.Developers reported plausible but incorrect field names and code that looked right on a quick scan but failed closer inspection.Source: §7.2 · pp. 11–12
  • Some users limit completions to short snippets or trigger suggestions manually when checking costs, interruptions, or distractions outweigh likely time savings.Building a mental model of when the tool will help takes time and may be wrong.Source: §7.2 · pp. 11–12

7.3. These tools are useful for boilerplate and code reuse

  • Developers find the tools especially useful for boilerplate, repetitive tasks, and common code that would otherwise require searching for examples.Reports also describe help with unfamiliar languages and APIs, and a significant velocity boost in greenfield projects with substantial boilerplate.Source: §7.3 · p. 13

8. The inadequacy of existing metaphors for AI-assisted programming

LLM-assisted programming resembles search, compilation, and pair programming, but each analogy misses important aspects of the experience. The authors argue it is a distinct way of programming with effects across many programming activities.

8.1. AI assistance as search

  • Search offers mixed-media results and often provenance, whereas LLM assistance returns editor-bound tokens and makes generated code’s provenance difficult to determine.The fixed-media experience can speed code insertion but loses search’s opportunities for learning, exploration, and discovery.Source: §8.1 · pp. 13–14

8.2. AI assistance as compilation

  • LLM assistance can reduce attention to syntax and APIs, but programmers must still understand, check, and adapt generated code, unlike compiler users.Current tools are also less deterministic and offer little feedback for improving the source prompt.Source: §8.2 · pp. 14–15
  • Natural-language prompts can span from describing an application to specifying an algorithm step by step, allowing movement across abstraction levels rather than one upward step.The authors describe this as teleporting the programmer to arbitrary rungs of the abstraction ladder.Source: §8.2 · p. 15

8.3. AI assistance as pair programming

  • The pair-programming analogy is superficial: programmers may shift between navigation and statement-level checking, while AI assistance lacks pair programming’s verbalization and shared-time focus.A preliminary comparison found more lines written with Copilot than in human pairs, but more were later deleted.Source: §8.3 · p. 16

8.4. A distinct way of programming

  • LLM-assisted programming is a distinct way of programming because search, compilation, and pair-programming metaphors each capture only part of its experience.It affects authoring, information foraging, debugging, refactoring, testing, documentation, maintenance, and learning.Source: §8.4 · pp. 16–17

9. Issues with application to end-user programming

End users may stand to benefit greatly from LLM assistance, but applying it to spreadsheet tasks raises challenges in specifying intent, checking and understanding code, automation’s effects, and whether users want code at all.

9. Issues with application to end-user programming

  • In GridBook, data analysts used natural-language utterances to synthesize spreadsheet formulas, with the utterances and formulas retained for editing.The study involved 20 analysts conducting exploratory data analysis.Source: §9 · p. 17

9.1. Issue 1: Intent specification, problem decomposition and computational thinking

  • End users may need repeated prompt refinement because requests can be underspecified, ambiguous, complex, or use domain phrases absent from the data context.Effective use also requires computational thinking, such as decomposing a task into computable subproblems; systems could guide users with examples, alternatives, or disambiguating questions.Source: §9.1 · p. 18

9.2. Issue 2: Code correctness, quality and (over)confidence

  • GridBook participants often judged formula correctness by eyeballing the result rather than reading code or testing rigorously, and some users may overestimate AI accuracy.End users may also struggle to recognize security, robustness, or performance problems in generated code.Source: §9.2 · p. 18

9.3. Issue 3: Code comprehension and maintenance

  • Participants found generated formulas hard to understand, which can hinder logic checking, customization, debugging, and reuse.The paper suggests synthesizing for readability or explaining code in terms of users’ tasks and context.Source: §9.3 · p. 18

9.4. Issue 4: Consequences of automation in end-user programming

  • Faster LLM-assisted work could leave end users with limited attention for testing or learning, raising questions about whether saved time can change those behaviors.The paper also asks how local experts’ roles may change if AI answers more of the questions users bring to them.Source: §9.4 · p. 19

9.5. Issue 5: No code, and the dilemma of the direct answer

  • Some spreadsheet users want an ad-hoc result rather than reusable code, while unfamiliar languages can be difficult for novices to understand.For data wrangling, Python’s extensive libraries may solve some user problems better than Excel formulas, despite the comprehension barrier.Source: §9.5 · p. 19

10. Conclusion

LLMs substantially expand automatic code generation, and commercial tools suggest a new way of programming. The conclusion highlights both broad changes to programming experience and unresolved challenges for non-expert end users.

10. Conclusion

  • LLM assistance transforms many aspects of programming, including planning, authoring, reuse, modification, comprehension, and debugging.The authors describe it as a new way of programming with challenges and opportunities for human-centric research, especially for non-expert end users.Source: §10 · p. 19

A. Experience report sources

The appendix lists sources used for the Section 7 quotations and analysis, though not every listed source is quoted directly.

A. Experience report sources

A.1. Blog posts and corresponding Hacker News discussions

This section lists six blog posts and associated Hacker News discussions about using GitHub Copilot and AI in development.

  • [1] Andrew Mayne describes building games and apps entirely through natural language using OpenAI’s code-davinci model.Source: §A.1 · p. 20
  • [2] Andrew Mouboussin describes building a no-code machine-learning model by chatting with GitHub Copilot.Source: §A.1 · p. 20
  • [3] Matt Rickard reports on one month of using GitHub Copilot.Source: §A.1 · p. 20
  • [4] Nutanc describes using GitHub Copilot to find the sentiment of tweets for a keyword in two minutes.Source: §A.1 · p. 20
  • [5] Tanishq Abraham writes about coding with GitHub Copilot.Source: §A.1 · p. 20
  • [6] Aleksej Komnenovic cautions against fully trusting AI in development work yet.Source: §A.1 · p. 20

Related papers

Loading 2208.06213v2…