PaperlayerView exact revision on arXiv

Source-linked AI summary

Vibe coding: programming through conversation with artificial intelligence

Advait Sarkar, Ian Drosos

arXiv:2506.23253v2cs.HC

TL;DR

Vibe coding raises questions about how programming changes when code is produced through interaction with generative AI rather than direct manipulation. This study provides a first empirical examination of the practice through curated think-aloud videos, finding iterative, selectively supervised workflows that redistribute rather than remove programming expertise.

  • Problem

    The paper addresses limited empirical understanding of vibe coding as an emerging programming practice and its implications for expertise, agency, control, and trust.

  • Method

    The authors conduct framework analysis of curated think-aloud videos from YouTube and Twitch, examining how programmers form goals, work, and deploy expertise.

  • Results

    Vibe coding involves iterative cycles of prompting, evaluation, debugging, and refinement, with strategic AI delegation, human oversight, blended prompting, and dynamic trust.

  • Takeaways & Limitations

    Programming expertise remains essential but is redirected toward context management, rapid code evaluation, bug resolution, and transitions between AI assistance and manual intervention.

  • Takeaways & Limitations

    The study analyses a small corpus of rapidly evolving, negotiated practice and includes no non-expert end-users.

Abstract

from arXiv · show

We examine "vibe coding": an emerging programming paradigm where developers primarily write code by interacting with code-generating large language models rather than writing code directly. We present the first empirical study of vibe coding. We analysed over 8 hours of curated video capturing extended vibe coding sessions with rich think-aloud reflections. Using framework analysis, we investigated programmers' goals, workflows, prompting techniques, debugging approaches, and challenges encountered. We find that vibe coding follows iterative goal satisfaction cycles where developers alternate between prompting AI, evaluating generated code through rapid scanning and application testing, and manual editing. Prompts in vibe coding blend vague, high-level directives with detailed technical specifications. Debugging remains a hybrid process combining AI assistance with manual practices. Critically, vibe coding does not eliminate the need for programming expertise but rather redistributes it toward context management, rapid code evaluation, and decisions about when to transition between AI-driven and manual manipulation of code. Trust in AI tools during vibe coding is dynamic and contextual, developed through iterative verification rather than blanket acceptance. Vibe coding is an evolution of AI-assisted programming that represents an early manifestation of "material disengagement", wherein practitioners orchestrate code production and manipulation, mediated through AI, while maintaining selective and strategic oversight.

1. Introduction and Background

Vibe coding emerged as an influential, evolving programming practice centered on delegating code production and manipulation to AI through natural-language interaction. This study provides an initial empirical account while treating the phenomenon as a temporally contingent snapshot.

  • Concept and origins: Vibe coding idealises disengagement from directly authoring, editing, and reading code, instead relying on agentic tools to perform these operations from natural-language descriptions.The Karpathy canon frames this as trusting AI-mediated code production for practical use.
  • Concept and origins: The programming community is actively negotiating what counts as vibe coding, including the roles of manual editing, keyboards, dictation, and non-agentic tools.The paper does not attempt to explain this broader sociological process.
  • Motivation: The paper studies vibe coding because programming offers an early view of how generative AI may affect knowledge work more broadly.The authors describe programming as unusually reflexive and positioned at the frontier of generative-AI tool development.
  • Study contribution: The authors present the first empirical analysis of vibe coding using framework analysis of curated think-aloud videos from YouTube and Twitch.The analysis addresses goals, workflows, prompting, debugging, challenges, expertise, and trust.
  • Scope: The study’s findings are preliminary because they draw on a small set of videos within a rapidly evolving landscape of tools and practices.The results should be read as a temporally contingent snapshot and a foundation for future comparison.

2. Method

The study uses framework analysis of curated online think-aloud videos to examine vibe coding as an emerging practice. Researchers progressively screened videos, developed inclusion criteria and analytical categories, and analysed five sessions in depth.

  • Data sources and selection: The researchers selected YouTube and Twitch videos because they provide access to observed actions and verbal reflections in emerging programming practices.At this stage, the study prioritised activity over public discourse about vibe coding.
  • Analysis method: Framework analysis was chosen to flexibly capture isolated events, event chains, concurrent events, and temporally disconnected but conceptually related activities.The method also accommodates both think-aloud information and observed video actions.
  • Scope: The dataset contained no sessions featuring users without programming expertise, so differences between programmers and non-programmers could not be analysed.This absence constrained the study’s ability to assess how non-experts encounter vibe coding challenges.
  • Analysis method: The final framework contained 9 top-level categories and 20 subcategories covering goals, intentions, workflow, prompting, debugging, challenges, expertise, trust, and definition and performance.Researchers independently analysed similarly sized portions of the dataset using qualitative notes and transcript excerpts.

3. Results

Across varied projects, vibe coding involved evolving goals, iterative interaction with AI, and negotiated movement between generated output and human judgment. Early generated choices could constrain later directions while also enabling exploratory possibilities.

  • Projects: The projects ranged from an animated explanatory diagram to deployed website features and a code-base chat application, spanning different domains and complexity levels.This variation provided examples of vibe coding beyond a single application type.
  • Goals and intentions: Vibe coding sessions combined expectations of full success, partial achievement, and exploratory use, with expectations sometimes changing during the process.Examples included accepting an outcome that reached “80% of the way” and pursuing an MVP rather than a production-ready system.
  • Goals and intentions: Most creators formed objectives before using AI, but iterative evaluation and dissatisfaction refined both project goals and prompting strategies.Users simultaneously developed intentions for the artefact and for communicating those intentions to AI.
  • Workflow and refinement: AI shaped final artefacts by filling implementation gaps, suggesting possibilities, and prompting users to accept, remove, or add features during evaluation.Testing revealed needs such as a stop button, while generated structures inspired additions such as animation.
  • Workflow and refinement: Context momentum created path dependence: earlier prompts and generated outputs steered later development toward trajectories that could become difficult to change.A model’s interpretation of a date-range request as a single-date picker became the project’s accepted direction.
  • Workflow and refinement: Context momentum could also support exploration, as satisfactory generated structures enabled developers to formulate new goals such as adding animations.Different satisfactory implementation choices might have led to different subsequent goals.

3.3. Workflow

Vibe coding proceeds through iterative goal-satisfaction cycles in which developers prompt AI, evaluate outputs, test applications, and refine or manually edit code. Effective workflows rely on deliberate context management, varied prompting strategies, and prompts ranging from broad directives to detailed specifications.

  • Workflow cycles: Developers repeatedly formulate goals, prompt AI, review generated code, accept or reject changes, test applications, and refine prompts or edit manually until goals are satisfied.The cycle may return to new goals or pivot when a sub-goal is completed or abandoned.
  • Preparation: Developers prepare prompts by selecting tools and models, customising system instructions, comparing model capabilities, and decomposing goals into smaller steps.These preparations reflect attention to integration fitness, speed, quality, cost, and project-specific technologies.
  • Workflow scope: Vibe coding occurs across projects begun from scratch, based on boilerplates, or continued within existing codebases.The observed workflow was not tied to one initial codebase condition.
  • Prompting practices: Prompting is supported by context management, named code elements, structured requirements, examples, error messages, documentation, and deliberate scope limitation.Developers also use external tools and system instructions to shape the model’s working context and reduce hallucinations.
  • Prompt granularity: Prompts span high-level directives and highly detailed implementation instructions, often combining vague aesthetic language with precise technical constraints.When broad prompts fail, developers commonly increase specificity by naming functions, variables, files, desired outcomes, or constraints.

3.5. Debugging

Debugging in vibe coding combines rapid code inspection, application testing, targeted prompting, and manual intervention. Programmers use expertise to identify likely problems, verify AI outputs against codebase structure and documentation, and choose between AI-assisted and traditional debugging.

  • Debugging process: Developers typically begin debugging with visual code inspection and application testing, then select more detailed strategies according to the issue encountered.Browser developer tools, console and network inspection, terminal analysis, and AI prompting all appear in the observed sessions.
  • Failure detection: AI-generated failures include hallucinated properties and noncompliance with instructions, requiring detection through error messages, code inspection, or testing.Observed issues included invalid properties, unwanted exports, and documentation mismatches.
  • AI assistance: AI-assisted debugging often involves pasting runtime errors into the coding tool and requesting fixes or documentation-guided corrections.This approach can bypass parts of manual debugging, although developers also refine prompts to address visual and functional discrepancies.
  • Manual intervention: Manual review and editing remain part of debugging, with developers inspecting diffs, forming hypotheses from errors and behaviour, testing them through tools, and applying targeted fixes.Model switching is also used occasionally when developers perceive different models as better suited to particular problems.
  • Rapid inspection: Programmers scan AI-generated code impressionistically rather than line by line, using diffs, structure, identifiers, comments, and key calls to assess likely correctness.Diff size and shape can support rapid acceptance or rejection, while suspicious details trigger closer inspection.
  • Expert verification: Expertise enables programmers to recognise suitable abstractions, cross-check APIs and parameters across files, and anticipate changes that the model has not proposed.Some experts can mentally visualise the resulting product from code structure without immediately running the application.

3.6. Challenges

Vibe coding creates technical, conceptual, and workflow challenges involving communication, model selection, tool limitations, and decisions about when to use AI or manual work.

  • Communication: Visual and abstract goals can be difficult to articulate in prompts, especially when creators cannot yet specify exactly what they want.This difficulty appeared in describing diagram arrows and conceptual product features.
  • Model selection: Choosing an appropriate model requires experience with differing capabilities and fit for particular workflows.Creators switched models after unsatisfactory results and associated different models with different task strengths.
  • Tool limitations: Tool-specific limitations can hinder recovery and integration, including difficulty reverting changes or incorporating code shown as chat output rather than file edits.These problems concern the surrounding development interface, not only generated code.
  • Workflow decisions: Deciding whether to prompt the AI or intervene manually adds metacognitive load, with observed developers switching approaches without consistently explicit criteria.Reviewing or rejecting changes can help developers retain control, while prompt refinement is often the first response to communication problems.
  • Prompt experimentation: Some creators experiment with social or emotional appeals in prompts, but the observed example still produced dissatisfaction with code quality.The strategy generated more code than the prior turn in one session.

3.7. Expertise

Expertise remains central to vibe coding but is redirected from direct code authorship toward tool selection, prompting, rapid evaluation, debugging, quality control, and transitions between AI and manual work.

  • Expertise deployment: Expertise is deployed throughout vibe coding, including selecting tools and models, evaluating outputs, diagnosing errors, controlling quality, and deciding when to edit manually.The role spans the full workflow rather than a single debugging or review stage.
  • Expertise redistribution: Programming expertise is redirected toward evaluating, guiding, and refining AI-generated solutions, making the programmer more director, reviewer, and editor than line-by-line author.Technical knowledge remains essential even when the programmer does not directly write every line.
  • Expanded expertise: Effective vibe coding combines traditional coding knowledge with AI expertise, product-management expertise, prompting skill, and understanding of user and product requirements.The observed competence includes model limitations, context windows, code quality, debugging, architecture, and feature formation.
  • Ambient competence: Some programmers may develop ambient competence: a feeling of being able to tackle tasks because an AI system might accomplish them with little effort.This observation is presented as a speculation about changing perceptions of capability.
  • Manual transitions: Developers switch to manual work when direct editing is more efficient, when generated code is close but needs refinement, or when debugging requires traditional intervention.Examples include one-line edits, autocomplete and search, unwanted features, environment files, and runtime errors.
  • Individual variation: The transition between AI and manual work varies across practitioners, with some relying heavily on re-prompting even during errors rather than fixing code themselves.One creator explicitly followed the Karpathy canon by avoiding manual debugging, contrasting with sessions that used traditional techniques.

3.8. Trust

Trust in vibe coding is granular, dynamic, and contingent on review. Developers build and calibrate trust through experimentation, verification, and adaptation rather than blindly accepting AI outputs.

  • Trust develops through interaction with AI and remains contingent on review, creating tension between efficiency and comprehension.Successful outputs can strengthen trust, while review and testing keep it calibrated.
  • Trust is strengthened by dramatic successful outputs but remains subject to continued verification and critical evaluation.Positive results reinforce confidence, while testing and review prevent blanket acceptance.
  • Developers mitigate overtrust by reviewing generated code, comparing it with documentation, testing frequently, and rejecting unmaintainable or invalid suggestions.YT21 and YT22 explicitly reject blind following and inspect outputs to remain in control.
  • Technical expertise shapes trust because experts can recognise overly complicated or problematic generated code and repair it.TW1 suggests less-experienced users may be unable to fix similarly complex outputs.

3.9. Defining and performing vibe coding

Vibe coding is presented as a flexible, conversational practice involving substantial delegation to AI without eliminating human review. Its streamed performances shape both the visible workflow and how the practice is interpreted.

  • Defining vibe coding: Vibe coding involves conversational interaction in which AI performs substantial coding work while programmers iteratively refine the resulting artefact.The practice is described as chatting with an application and having AI create features that users then improve.
  • Defining vibe coding: Vibe coding occupies a higher AI-reliance position than traditional AI-assisted coding but remains neither entirely hands-off nor free of human intervention.The distinction is significant delegation of code creation and modification rather than isolated completion or discrete tasks.
  • Defining vibe coding: Developers may accept outputs quickly early in development but become more critical as projects progress and requirements become more consequential.The same practitioner can describe both embracing AI and rejecting blind following, reflecting evolving reliance across a session.
  • Performing vibe coding: Streaming encourages enthusiastic demonstrations of AI capability, which can amplify perceived effectiveness and potentially exaggerate it for audiences.Creators highlight ease and speed, while the performance context shapes how results are presented.
  • Performing vibe coding: Audience interaction introduces tangents and spontaneous explanations, producing workflow dynamics that may not occur in private coding.Chat responses can redirect activity, although they can also reveal reasoning about context windows and instruction length.
  • Performing vibe coding: Performance pressures can encourage procedural shortcuts and video-friendly stopping points that preserve narrative momentum over complete testing.Examples include skipping an API test and ending development at milestones selected for video length.
  • Performing vibe coding: Despite performative emphasis, the videos still show debugging, manual intervention, and commentary that can help creators maintain control and momentum.The observed sessions therefore include realistic workflow elements alongside impression-management pressures.

4. Discussion

The discussion frames vibe coding as a more hands-off, conversational evolution of AI-assisted programming that changes how expertise and material engagement are deployed. Programmers disengage from line-by-line code production but continue selectively evaluating and manipulating code through AI-mediated workflows.

  • Vibe coding shifts interaction toward trusting AI to handle more coding and error resolution, while retaining human judgment and expertise.The practice leans into conversational generation but does not eliminate strategic oversight.
  • Prompts mix vague goals with detailed technical specifications, complicating how programmers communicate intent to language models.The mixed granularity differs from earlier practices that more often relied on explicit comments and matched abstraction levels.
  • Programmers evaluate generated code through rapid scanning, pattern recognition, and targeted inspection rather than routinely reviewing every line deeply.These judgments depend on mental schemas for coding patterns, frameworks, and the relevant codebase.
  • Expertise is redirected from direct authorship toward selecting tools, managing context, evaluating outputs, and deciding when to switch between AI-driven and manual work.The programmer acts more as director, reviewer, and editor while technical knowledge remains essential.
  • Material engagement: The paper treats vibe coding as material disengagement in which AI mediates the generation and modification of large code sections.The programmer steps back from direct manipulation, while code remains part of the workflow through selective review, editing, and debugging.
  • Material engagement: AI-generated code, commentary, and errors help refine programmers’ intentions, making interaction with the AI interface a new form of material engagement.Unexpected outputs and failures function as resistances that prompt adjustments and strategy refinement.
  • Material engagement: Direct code disengagement may reduce the deep understanding and skill formation associated with grappling firsthand with syntax, structure, and debugging resistance.This potential loss is identified as a consequence of mediating code work through an AI agent.
  • Material engagement: Selective engagement with code persists through diff scanning, keyword inspection, visual cues, and tactical manual writing, editing, and debugging.These practices still require substantial expertise in manipulating and interpreting the underlying code material.

5. Conclusion

This study provides the first empirical analysis of vibe coding, showing it as an AI-mediated programming practice that retains human oversight and expertise. It offers a foundation for future research on evolving human-AI workflows.

  • The study presents the first empirical analysis of vibe coding, where developers author and edit code through natural-language interaction with code-generating AI.
  • Vibe coding proceeds through iterative goal-satisfaction cycles involving prompting, evaluation, debugging, and refinement, with strategic delegation to AI and targeted human intervention.
  • Programming expertise is redistributed toward context management, rapid code evaluation, bug resolution, and decisions about switching between AI assistance and manual intervention.
  • Vibe coding exemplifies material disengagement, in which practitioners orchestrate content production through AI intermediaries while still requiring substantial expertise in the underlying material substrate.
  • The research provides a foundational reference for future investigations into human-AI workflows in programming and knowledge work as vibe coding practices evolve.

A. Video sources

The supplied material identifies the sources table and its continuation but does not provide the videos’ individual entries or source characteristics.

  • The source material is organized as a table of videos included in the analysis, but the supplied passages do not enumerate those videos.
  • The sources table continues across multiple table segments, without supplied details about the entries represented in those segments.

B. Analysis framework

The analysis framework organizes the study around questions concerning vibe coding’s definition, workflow, expertise, challenges, trust, and public performance. It was designed to capture varied and temporally distributed phenomena in an emerging practice.

  • Analysis framework: It investigates goals, workflows, debugging, challenges, expertise, trust, and transitions between AI assistance and manual work.
  • Expertise: The framework asks how experts deploy developer knowledge, system understanding, technical-ecosystem knowledge, and technology-oriented attitudes during vibe coding.
  • Trust: Trust-related questions address how users develop trust in AI outputs and the risks of overtrust or overreliance.
  • Definition of vibe coding and performance: The framework examines how programmers define vibe coding and distinguish it from other AI-assisted programming activities.
  • Definition of vibe coding and performance: The framework considers how YouTube and Twitch performance may shape tool choice, prompting strategies, and users’ framing of their experiences.
Loading 2506.23253v2…