Is AI the better programming partner? Human-Human Pair Programming vs. Human-AI pAIr Programming

• Pair programming began as two programmers sharing one computer, with a driver coding and a navigator planning, reviewing, debugging, and suggesting alternatives.Source: §1 · p. 1
• pAIr programming pairs a programmer with an LLM-based AI, prompting questions about comparable performance, contexts, and interaction with human pairs.Source: §1 · p. 1
• The review compares application contexts, methods, and tasks, success measures, and moderators such as expertise-related compatibility.Source: §1 · p. 1
• Both paradigms show mixed results, while pAIr studies lack comprehensive measures and have largely unexplored success factors.Source: §1 · p. 1
• Human-human pairing challenges such as compatibility and communication suggest pAIr designs could adapt expertise levels and communication styles.The paper also encourages research on student-AI pairing and its potential challenges.Source: §1 · pp. 1–2

• The comparison focuses on industry and education and retains the original single-device pair-programming definition, excluding distributed and computer-mediated modes.Source: §2 · p. 2
• Most pAIr studies evaluate Copilot through case studies or experiments with experienced industry programmers.These studies often try to mimic realistic professional development tasks, but non-invasive field observation is lacking.Source: §2 · p. 2
• Student use has received little study, and pAIr research remains mostly laboratory-based, lacking large-scale studies and classroom deployment.One cited experiment involved 69 novice students aged 10–17 completing 45 Python tasks.Source: §2 · p. 2
• Existing comparisons use human-AI versus human-human or human-solo groups, but no study uses all three comparison groups.Source: §2 · p. 2
• Compared with human-human research, pAIr studies lack realistic workspace or classroom deployment, and larger samples are desirable.The review uses comparisons of variables and measurements to identify further gaps in pAIr studies.Source: §2 · p. 2

3.1 Quality

• Industry reviews report quality gains, but Hannay et al. also describe trade-offs: higher quality requires more effort, and shorter duration accompanies lower quality.Alves De Lima Salge and Berente report improved code quality, productivity, and learning outcomes.Source: §3 · p. 2

3 MIXED OUTCOMES

• Human-AI studies report mixed quality and productivity findings, but evidence is too limited and outcomes too differently measured to conclude effectiveness.Existing human-AI studies mainly focus on quality, productivity, and satisfaction.Source: §3 · p. 2

3.2 Productivity

• Productivity aggregates different efficiency measures, but added-line counts may misrepresent it because Copilot's tab-to-accept interaction can increase lines.The review groups productivity, duration, and effort as efficiency outcomes; perceived productivity is distinct from actual productivity and correlates most strongly with AI-code acceptance rate [90].Source: §3.2 · pp. 2–3

3.3 Learning

• Human-human educational studies assess learning through grades, exam scores, passing rates, retention, and qualitative measures of higher-order thinking skills.Reported educational benefits also include student confidence in solutions.Source: §3 · p. 2, §3.3 · p. 3

3.4 Cost

• No study experimentally characterizes human-AI pair-programming costs, despite reported Copilot-code debugging challenges and evidence that its bugs can be easier to debug than human errors.The debugging challenges motivate a hypothesis of additional effort and reduced task-solving effectiveness, not an experimentally established cost.Source: §3.4 · p. 3

3 MIXED OUTCOMES

• Human-human pair-programming reviews report mixed outcomes across quality, productivity, satisfaction, learning, and cost, with individual studies sometimes contradicting aggregate trends.Meta-analyses can show overall trends and significant effects even when individual studies report contradictory outcomes.Source: §3 · p. 2, §3.4 · p. 4

4 MODERATORS

• Human-human moderators include task type and complexity, compatibility, communication, collaboration, and logistics; their effects on pAIr programming remain insufficiently examined.When the factors work well, pair programming can improve error detection, problem solving, code review, code quality, and knowledge sharing; when they do not, costs and delays can follow.Source: §4 · p. 4

4.1 Task Types & Complexity

• Task complexity changes the trade-off between duration, quality, and effort, with its effects also depending on pair expertise.Debugging tasks were less satisfying and efficacious than comprehension or refactoring tasks; higher complexity could raise quality but require considerably greater effort.Source: §4.1 · p. 4

4.2 Compatibility

• Similar expertise often supports productivity and learning, but very small or large skill gaps can also create problems.Novice pairs showed larger productivity gains than expert pairs, while large gaps could reduce satisfaction and quality benefits; novice-expert pairs could leave novices disengaged or less confident.Source: §4.2 · p. 4

4.3 Communication

• Human-AI interaction takes time for suggestion checking and prompt crafting, but lacks a human co-verifier and has not been evaluated as pair communication.Copilot users spent 22.4% of time thinking and verifying suggestions and 11.56% crafting prompts; human-human pairs spend about one-third of time communicating.Source: §4.3 · pp. 4–5

4.4 Collaboration

• Free-riding can reduce satisfaction and learning, while over-reliance on AI may hinder student learning but has not been formally evaluated.Human pairs may switch driver and navigator roles; with Copilot, humans still need to switch between thinking and writing and remain responsible for understanding code at statement level.Source: §4.4 · p. 5

4.5 Logistics

• Scheduling, training, evaluation, and accountability add logistical costs or unresolved questions, and pAIr moderators need empirical testing.Human-AI pair programming may avoid scheduling difficulties but raises questions about accountability, collaboration training, bias, trust, and technical limitations.Source: §4.5 · p. 5

5.1 LLM, Your pAIr Programmer?

• Researchers disagree on whether LLM-assisted programming is pair programming, and the paper argues that comparing the approaches can inform improvements to LLM-based tools.Some characterize the analogy as superficial because productive human communication may be absent; others note similarities in outcomes and a need to examine moderators.Source: §5.1 · p. 6
• The paper uses “human-AI pAIr programming” for a human and AI working on the same device and task, while acknowledging that Copilot and human partners’ moderators are less examined.The review mostly covers the VSCode Copilot extension, although other tools may support communication or offer additional functionalities.Source: §5.1 · p. 6

5.2 LLM, A Better pAIr Programmer?

• Human-AI pair programming research needs broader evaluation measures and studies that compare human-human, human-AI, and solo programming on the same task.The review describes human-human measures as varied while current human-AI evaluation remains limited.Source: §5.2 · p. 6
• A key research question is whether AI can be configured to generate collaborative tasks and adapt their complexity to programmers.LLMs have been used to generate instructional materials and break problems into subquestions, but the proposed task-design use remains open.Source: §5.2 · p. 6

5.3 LLM, Students’ pAIr Programmer?

• For student-AI pairs, future work should determine whether AI expertise should match students, exceed it modestly, or be paired with a tutor agent.The discussion weighs potential learning gains against concerns about novice pairs lacking expertise and novice-expert pairs lowering novice self-esteem.Source: §5.3 · p. 8
• Student-AI designs should address over-reliance and over-helping while supporting code review, debugging, testing, explanation, and communication skills.Studies report that novices may rely on and trust Copilot-generated code too much; academic integrity and changing perceptions of learning also remain concerns.Source: §5.3 · p. 8

6 CONCLUSION

• Current research does not give a clear answer about the efficacy of human-AI pair programming.Both human-human and human-AI pair programming have benefits and challenges.Source: §6 · p. 8
• Human-human pair programming research can guide pAIr study designs, outcome measures, and analysis of moderators.Suggested priorities include more realistic observations, valid quality and productivity measures, further cost investigation, and moderators such as compatibility and communication.Source: §6 · p. 8
• More valid and comprehensive pAIr measurements, direct comparisons between paradigms, and research on supporting LLM-assisted programming remain needed.Source: §6 · p. 8