PaperlayerView exact revision on arXiv

Source-linked AI summary

What Will Remain Human in Software Architecture? A Focus Group Report

Uwe van Heesch, Olaf Zimmermann, Christian Kohls

arXiv:2609.30334v1cs.SEcs.AI

TL;DR

Autonomous development agents are taking on broader architectural work, raising questions about what remains human and what new responsibilities emerge. This focus-group report explores practitioner perspectives and identifies human accountability, harness engineering, cognitive debt, and unresolved harness-evaluation questions as key findings.

  • Problem

    Autonomous agents increasingly participate in architecture decisions, raising questions about practitioners’ roles and how AI-assisted system creation should be governed.

    Source: §1 · p. 2

  • Method

    The report explores these questions through a EuroPLoP focus group, informed by three earlier exploratory interviews.

    Source: §1 · p. 2

  • Results

    The findings identify architectural decisions and accountability as human responsibilities, harness engineering as an emerging practice, and cognitive debt as a risk of accepting AI-assisted decisions without full engagement.

    Source: §1 · p. 2, §7 · p. 18

  • Takeaways & Limitations

    How to evaluate, test, and evolve the system that builds the system remains unanswered in current software engineering practice.

    Source: §7 · p. 19

    The report is an experience report rather than a formal study, and it frames validity threats across four standard dimensions.

    Source: §6 · p. 17, §7 · p. 19

Abstract

from arXiv · show

AI development agents are increasingly used to support and partially automate software architecture tasks. To explore how practitioners perceive this shift, specifically what changes, what remains, and what new responsibilities emerge, we conducted a focus group at the 31st European Conference on Pattern Languages of Programs, People, and Practices (EuroPLoP 2026). Twenty-two participants from industry and academia discussed current practices, trust and validation strategies, the boundaries of AI autonomy, governance challenges, and implications for education. Among others, we found broad consensus that architectural decision-making, accountability, and the authoring of architectural guardrails remain fundamentally human tasks. A central emergent concept was harness engineering: the discipline of building the system that governs AI-assisted system creation, comprising validation mechanisms, knowledge lay- ers, and company-specific standards. The participants agreed that criticality, understood as the combination of uncertainty and cost of change, serves as the universal criterion for calibrating human oversight. A further concern was cognitive debt: the progressive erosion of human understanding of the system when AI-assisted decisions are accepted without full intellectual engagement. In this report, we present the findings of the focus group and describe directions for future work.

C. Kohls, TH Köln, Germany

The manuscript identifies its accepted-conference-paper status and lists the paper’s research themes.

C. Kohls, TH Köln, Germany

  • The listed keywords span software architecture, AI agents, harness engineering, cognitive debt, and human-AI collaboration.They also include focus groups and architectural decision-making.Source: C. Kohls, TH Köln, Germany · p. 1
  • The manuscript was accepted after peer review at EuroPLoP 2026 but is not the version of record.The authors ask readers to cite the forthcoming proceedings version.Source: C. Kohls, TH Köln, Germany · p. 1

1 Introduction

The introduction frames autonomous development agents as a shift from code completion toward broader architectural work, raising questions about architects’ future roles. The report uses a focus group to explore practitioner perspectives and identifies harness engineering, cognitive debt, and validation as central themes.

1 Introduction

  • Autonomous agents can propose, implement, and modify architecture decisions, prompting questions about the future role of human architects.The introduction contrasts these agents with earlier code-completion tools.Source: §1 · p. 2
  • The introduction describes a shift from implementation toward defining intent, constraints, quality, and validation, with delegation requiring explicit configuration and guardrails.Participants characterize the relationship with AI as increasingly like subcontracting.Source: §1 · p. 2
  • The report explores practitioner and researcher perspectives through a EuroPLoP focus group informed by three prior exploratory interviews.The interviews were part of a larger research effort but are not reported in this paper.Source: §1 · p. 2
  • Harness engineering and cognitive debt emerge as central findings, while validation of AI-generated artifacts is identified as a key challenge across architectural work.Cognitive debt is described as a risk when practitioners delegate without full intellectual engagement.Source: §1 · p. 2
  • The report presents focus-group experience rather than a formal empirical study and says the data will feed into a larger study.That larger study combines the exploratory interviews with the focus-group findings.Source: §1 · p. 2

2 Background and Related Work

The background connects architecture decision practices and human-AI collaboration research to the focus group’s findings. Across six related-work areas, it positions harness engineering as a way to govern AI-assisted system creation, distinct from governing the system itself.

2.1 Architecture Decisions

  • Participants report using AI to generate architecture decision records while retaining human judgment over the underlying rationale.This finding makes both lightweight and comprehensive decision-documentation approaches relevant.Source: §2.1 · p. 3

2.2 Human-AI Collaboration in Software Engineering

  • Prior studies describe AI interaction modes and trust calibrated by task complexity, alongside short-term velocity gains and potential long-term code complexity,,.These findings provide context for practitioners’ trust calibration in architecture proposals.Source: §2.2 · pp. 3–4

2.3 AI for Software Architecture

  • The focus group adds practitioner accounts of harness engineering, cognitive debt, and daily trust calibration to gaps identified in a review of AI support for architecture practice [15].The review synthesized 51 studies and identified challenges including context-aware reasoning and long-term evolution.Source: §2.3 · p. 4

2.4 AI Configuration and Context Engineering

  • Prior work describes configuration artifacts for steering agents, while participants’ harnesses are human-authored, version-controlled specifications governing project-level agent behavior,,.The artifacts encode project constraints and architectural decisions.Source: §2.4 · p. 4

2.5 Automated Architecture Governance

  • Harness engineering extends automated governance from enforcing constraints on a system to governing the AI-assisted process that creates it.The related work describes fitness functions and conformance checks as ways to make architectural intent explicit and enforceable.Source: §2.5 · p. 5

2.6 Community Discussions on the Future of Software Architecture

  • The focus group complements community discussions of architecture knowledge loss and human validation with practitioner narratives and harness engineering as an emergent discipline [19].The cited discussions also raise concerns about human control and certification of AI-assisted decisions.Source: §2.6 · p. 5

3 Focus Group Setup and Participants

The study used a 90-minute, moderated plenary-style focus group with participants from industry and academia. Researchers synthesized the transcript and session notes, validated thematic assignments collaboratively, and conducted member-checking.

3.1 Setup

  • The 90-minute EuroPLoP session used a circular seating arrangement, three researchers, and six prepared themes, but discussion flowed naturally between topics.With 22 participants, it functioned as a moderated plenary rather than a classical focus group with breakouts; 19 contributed actively.Source: §3.1 · pp. 5–6

3.2 Participants

  • Eighteen of 22 participants returned the questionnaire, representing industry and academia, with software-development experience ranging from 3 to 30 years.The 12 academic and six industry respondents reported varied AI tool use and professional contexts.Source: §3.2 · pp. 6–7

3.3 Data Collection and Analysis

  • Analysis synthesized the corrected transcript and researcher notes by discussion theme, with all three researchers validating assignments and five participants member-checking the reported findings.The authors characterize this as structured synthesis, not formal qualitative coding; the audio was transcribed and manually corrected for technical terms and proper nouns.Source: §3.3 · p. 7

4 Results

Participants described AI use across architecture work, from documentation and exploration to implementing architectural changes. They emphasized that human judgment remains central to consequential decisions, accountability, guardrails, and oversight, while harness engineering and cognitive debt emerged as new concerns.

4.1 Current Use of AI Tools for Architecture Tasks (Part A)

  • AI supports architecture work from ADRs and context gathering to alternative exploration, threat analysis, legacy documentation, prototypes, and direct architectural changes.Participants also said ADRs can serve as persistent agent memory; they reported that detailed plans improve outputs and existing code helps establish architectural style.Source: §4.1 · pp. 7–8

4.2 Trust and Validation of AI Proposals (Part B)

  • Trust is calibrated to decision criticality: low-consequence proposals may need less understanding, while consequential decisions prompt thorough verification and experimentation.Participants considered existing CI/CD, tests, and code-quality tools an appropriate quality backbone and also described cross-checking outputs with multiple models.Source: §4.2 · p. 9

4.3 Tasks that Remain Fundamentally Human (Part C)

  • Architectural decisions, accountability, and precise architectural guardrails remain human responsibilities, even when AI prepares options or implements changes.Participants also connected human oversight to organizational context that AI does not know, including team skills, strategic goals, and company constraints.Source: §4.3 · pp. 9–10
  • Participants raised cognitive debt as erosion of understanding when AI-assisted decisions are accepted without full engagement, and saw programming skills as more vulnerable than architectural thinking.They also described selective oversight through observable, well-contained components and concern that AI-mediated help can disrupt senior-to-junior mentoring.Source: §4.3 · pp. 10–11

4.4 New Responsibilities and Governance (Part D)

  • Harness engineering is the system governing AI-assisted system creation, combining company knowledge and standards with validation, observability, and hybrid checks.Participants said the harness must evolve with AI capabilities and organizational standards, while direct, output-independent assessment of harness quality remains an open problem.Source: §4.4 · pp. 11–12
  • Participants distinguished casual “demo software” from business-critical and acute “quality software,” voicing concern about trust-sensitive applications being created without professional engineering.They suggested engineering effort may shift toward building and maintaining the harness; the reported legacy-modernization story was explicitly unverified.Source: §4.4 · p. 12

4.5 Autonomous Decisions and Accountability (Part E)

  • Participants maintained that a human must remain accountable, while questioning whether one person can oversee a large AI-managed system; proposed alternatives included vendor liability or sufficient oversight staff.They also described architecture as shifting to the harness when end users bypass explicit architecture, alongside an expanding span of architect responsibility.Source: §4.5 · pp. 12–13

4.6 Implications for Education and Skill Development (Part F)

  • Participants said education should retain computing fundamentals and architectural competencies while adding precise specification, harness engineering, validation, and resilience.They suggested reducing breadth of programming-language exposure, while treating harness maintenance and understanding AI outputs as continuing competencies.Source: §4.6 · pp. 13–14

5 Discussion and Reflection

The discussion frames criticality as the criterion for calibrating AI autonomy and identifies harness engineering and cognitive debt as central themes. Participants describe unresolved tensions between short-term productivity and capabilities that may be difficult to recover.

5.1 Cross-Cutting Themes

  • Criticality—uncertainty combined with cost of change—sets the oversight level: low-criticality work permits more AI autonomy, while high-criticality work demands human control and full understanding.Participants applied this criterion to individual decisions, whole systems, and the choice of development approach; the framing was accepted without dissent.Source: §5.1 · p. 14
  • The subcontractor metaphor applies familiar trust, verification, and management practices to AI, while precise specifications become the primary human artifact as code generation is automated.Participants reported that working with AI improved their specification skills because imprecision leads directly to wrong outputs.Source: §5.1 · pp. 14–15
  • Participants saw both continuity in sound engineering principles and novelty in system structure and where architectural decisions are made.One participant argued AI could let a single architect achieve work previously requiring a team, but others expressed skepticism about enterprise adoption.Source: §5.1 · p. 15
  • Cognitive debt accumulates when people accept plausible AI decisions without understanding them, engage too briefly to retain knowledge, or routinely approve agent requests without conscious evaluation.These mechanisms can erode an architect’s coherent understanding of system structure, constraints, and trade-offs.Source: §5.1 · p. 15

5.2 Harness Engineering as Emergent Discipline

  • Harness engineering is the socio-technical system governing AI-assisted software creation, encoding organizational standards and controlling context, constraints, and output validation.It operates above the delivery pipeline and output conformance checks; its scope and design depend on software criticality and type.Source: §5.2 · p. 16
  • The harness connects governance-layer engineering, reusable company knowledge, and architectural decisions, which may reside in the harness when individual applications become disposable.The discussion also raised the need for LLM-agnostic architectures.Source: §5.2 · p. 16

5.3 Unresolved Tensions

  • Unresolved tensions pit AI efficiency against skill preservation, bounded-risk acceptance against full comprehension, regeneration against maintainability, and adequate solutions against exploring the full solution space.Participants did not resolve these tensions; criticality was their recurring guide for choosing where to operate on each spectrum, and whether the dilemmas are temporary remains unclear.Source: §5.3 · pp. 16–17

6 Limitations and Threats to Validity

The authors identify limits to the focus group’s representativeness, depth, and analysis. The findings should be read as an interpretation of a bounded discussion, not as representative evidence about the profession.

6.1 External Validity

  • Because participants came from one architecture-aware, pattern-oriented conference community, the findings reflect expert perspectives rather than the profession at large.The authors note that less architecture-engaged practitioners and domains with less advanced AI adoption may be underrepresented.Source: §6.1 · p. 17

6.2 Internal Validity

  • The plenary format may over-represent vocal participants, and 22 participants’ contributions were necessarily brief, leaving some topics unexplored in depth.The 90-minute session gave Parts C and D disproportionately more time than Parts E and F.Source: §6.2 · pp. 17–18

6.3 Construct Validity

  • The interview-derived discussion guide may have underexplored topics absent from the small set of exploratory interviews.The authors note that grounding questions in prior empirical data also strengthened their relevance.Source: §6.3 · p. 18

6.4 Reliability

  • The findings are a three-researcher structured synthesis, not formal qualitative coding, and are best understood as the authors’ interpretation of the discussion.Five participants confirmed the reported findings through member-checking; the moderator’s framing and follow-ups also influenced discussion direction.Source: §6.4 · p. 18

7 Concluding Summary and Future Work

The focus group identifies enduring human responsibilities and criticality-calibrated oversight alongside harness engineering and cognitive debt. It also sets out unresolved research questions and notes that a formal qualitative analysis will be reported separately.

7 Concluding Summary and Future Work

  • Architectural decision-making, accountability, and guardrail authorship remain human responsibilities; uncertainty and cost of change calibrate human oversight.The group distinguishes low-criticality applications, which may need minimal human involvement, from trust-critical systems requiring rigorous professional engineering.Source: §7 · p. 18
  • Harness engineering is the discipline of governing AI-assisted software creation through validation mechanisms, knowledge layers, and organizational standards.The group identifies it as a new discipline.Source: §7 · p. 18
  • Cognitive debt is the progressive erosion of human understanding when practitioners accept AI-assisted decisions without full intellectual engagement.Source: §7 · p. 18
  • Validation is key to success and harder for architecting than coding; open questions include harness components, independent quality evaluation, evolution, and links to established practices.Source: §7 · pp. 18–19
  • Evaluating, testing, and evolving “the system that builds the system” remains unanswered in current software engineering practice.Source: §7 · p. 19
  • This experience report is not a formal study; a separate report will present planned thematic coding, researcher validation, and triangulation with exploratory interviews.Source: §7 · p. 19
  • Future work includes longitudinal study of architectural competence and unresolved questions about accountability, trust-critical systems, mentoring, and preserving solution-space breadth.Source: §7 · p. 19
  • One participant summarized the group’s sentiment as “software development is dead – long live harness engineering.”Source: §7 · p. 19
Loading 2609.30334v1…