PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Human-Bot Collaborative Software Architecting with ChatGPT

Aakash Ahmad, Muhammad Waseem, Peng Liang, Mahdi Fehmideh, Mst Shamima Aktar, Tommi Mikkonen

arXiv:2302.14600v1cs.SEcs.AI

TL;DR

ACSE needs better support for coordinating architectural knowledge, automation, and expertise across complex software development. This paper presents a preliminary case study in which a novice architect collaborates with ChatGPT across architectural analysis, synthesis, and evaluation for a services-driven application. The results indicate that ChatGPT can support and sometimes lead architecting, but human oversight remains necessary.

  • Problem

    ACSE faces process, socio-technical, and expertise challenges, while ChatGPT’s role as a DevBot for end-to-end architecting remains insufficiently investigated.

  • Method

    The study uses a process-centric, scenario-based case study in which an architecture story is given to ChatGPT for collaborative analysis, synthesis, and evaluation of a microservices-driven application.

  • Results

    ChatGPT articulated requirements, specified models, recommended and applied tactics and patterns, and developed evaluation scenarios for the case study.

  • Takeaways & Limitations

    ChatGPT can support and sometimes lead ACSE, while collaborative architecting still requires architects’ oversight and decision support.

  • Takeaways & Limitations

    The study is preliminary and leaves empirical validation of collaborative architecting for future work.

Abstract

from arXiv · show

Architecting software-intensive systems can be a complex process. It deals with the daunting tasks of unifying stakeholders' perspectives, designers' intellect, tool-based automation, pattern-driven reuse, and so on, to sketch a blueprint that guides software implementation and evaluation. Despite its benefits, architecture-centric software engineering (ACSE) inherits a multitude of challenges. ACSE challenges could stem from a lack of standardized processes, socio-technical limitations, and scarcity of human expertise etc. that can impede the development of existing and emergent classes of software (e.g., IoTs, blockchain, quantum systems). Software Development Bots (DevBots) trained on large language models can help synergise architects' knowledge with artificially intelligent decision support to enable rapid architecting in a human-bot collaborative ACSE. An emerging solution to enable this collaboration is ChatGPT, a disruptive technology not primarily introduced for software engineering, but is capable of articulating and refining architectural artifacts based on natural language processing. We detail a case study that involves collaboration between a novice software architect and ChatGPT for architectural analysis, synthesis, and evaluation of a services-driven software application. Preliminary results indicate that ChatGPT can mimic an architect's role to support and often lead ACSE, however; it requires human oversight and decision support for collaborative architecting. Future research focuses on harnessing empirical evidence about architects' productivity and exploring socio-technical aspects of architecting with ChatGPT to tackle emerging and futuristic challenges of ACSE.

I. INTRODUCTION

ACSE faces process, expertise, and socio-technical challenges, while ChatGPT offers a potential DevBot for human-bot collaborative architecting. This preliminary study investigates ChatGPT’s ability to support architectural analysis, synthesis, and evaluation under human oversight.

  • ACSE combines architectural knowledge, languages, tools, and human decisions to guide software implementation, validation, and maintenance.
  • Lack of automation and architect expertise can hinder development of complex and emergent software systems, creating demand for process- and tool-based decision support.
  • ChatGPT can generate architectural requirements and other software artifacts despite not being specifically developed for software engineering.
  • The study investigates whether ChatGPT can process an architect-conversed architecture story and perform architectural analysis, synthesis, and evaluation collaboratively.
  • The preliminary case study uses a process-centric, scenario-based method to examine ChatGPT-enabled architecting of a microservices-driven application.
  • The study identifies both the potential and perils of ChatGPT-assisted ACSE, including concerns about ethics, governance, and socio-technical constraints.
  • Its proposed foundation supports future empirical research on ChatGPT’s capabilities and architects’ productivity in collaborative architecting.

II. RESEARCH CONTEXT AND METHOD

The paper frames collaborative architecting as an extension of systematic ACSE, using DevBots to combine human architectural rationale with automated assistance. ChatGPT is positioned as a conversational bot that can help create architectural artifacts under human supervision.

  • A. Human-Bot Collaborative Architecting: Software architecture abstracts implementation complexity through components and connectors that blueprint applications, services, and systems.
  • A. Human-Bot Collaborative Architecting: An architecting process separates concerns into activities such as architectural analysis, synthesis, and scenario-based evaluation.
  • A. Human-Bot Collaborative Architecting: DevBots are AI-driven conversational agents or recommender systems that automate or add intelligence to software engineering processes.
  • A. Human-Bot Collaborative Architecting: Prior DevBot applications were limited mainly to answering questions or recommending solutions for architectural erosion and maintenance.
  • A. Human-Bot Collaborative Architecting: ChatGPT-based collaboration extends beyond questions and recommendations by translating natural-language requirements into architectural requirements, models, and evaluation scenarios with human supervision.

B. Research Method

The research method organizes collaborative architecting around an architecture story and three process activities: analysis, synthesis, and evaluation. Empirical validation remains an extension and future phase rather than part of the existing study scope.

  • Research Method: The methodology has three phases: developing the architecture story, enabling collaborative architecting, and conducting empirical validations.
  • Phase 1 - Developing the Architecture Story: An architecture story narrates the envisioned software’s functionality, desired quality attributes, and constraints in natural language before being supplied to ChatGPT.
  • Phase 2 - Enabling Collaborative Architecting: Architectural analysis uses the story to articulate requirements through ChatGPT generation, architect specification, or continuous dialog-based refinement.
  • Phase 2 - Enabling Collaborative Architecting: Architectural synthesis consolidates ASRs into a UML architecture model representing structural decomposition and runtime scenarios, refined with tactics and patterns.
  • Phase 2 - Enabling Collaborative Architecting: Architectural evaluation incrementally validates the synthesized architecture against ASRs and scenarios using the Software Architecture Analysis Method.
  • Phase 3 - Conducting the Empirical Validations: Empirical validation is described as an extension and future-work phase for studying ChatGPT’s role and associated socio-technical issues.

III. CASE STUDY ON COLLABORATIVE ARCHITECTING

The case study demonstrates collaborative architecting for CampusBike, a web and mobile application supporting campus bicycle rental and related services. A motivated novice engineer participates in designing and developing the system.

  • Case Study Context: CampusBike supports registration, bike discovery, reservations, payments, and usage reporting through browser and app interfaces.
  • Case Study Context: The application targets eco-friendly campus mobility by enabling visitors to use pay-per-use bicycles hourly or daily.
  • Case Study Context: The case study’s architect is a motivated novice engineer with working knowledge of UML, patterns, programming, and scripting languages.
  • Architecture Story: The architecture story includes location-based bike discovery within a stated 500-meter proximity and time-specific reservation after payment.

A. Formulating the Architecture Story

The architecture story narrates the envisaged software, its core functionality, and constraints in natural language, then serves as input for iterative human-bot refinement of architectural requirements. The case study uses architect queries and ChatGPT responses to specify and refine functionality, quality, and constraints.

  • A. Formulating the Architecture Story: An architecture story describes the envisaged software, core functionality, and constraints in natural language before architectural analysis.The architect derives the story by analysing the software domain and identifying operational scenarios.
  • A. Formulating the Architecture Story: The architecture story is fed to ChatGPT as the foundation for specifying architectural significant requirements and constraints.The requirements include required functionality, desired quality, and constraints for the CampusBike software.
  • A. Formulating the Architecture Story: The process begins with a natural-language architecture story and proceeds to requirements formulation and refinement.The cited figures identify the broader collaborative architecting process and the requirements-formulation stage.
  • B. Architectural Analysis: ChatGPT can outline ASRs and constraints, but the case study reports that its initial expressions required architect refinement.The final requirements were produced through an iterative dialogue between the architect and ChatGPT.
  • B. Architectural Analysis: ASRs are iteratively refined through human-bot dialogue into a final list of functionality, quality, and constraints.This dialogue is presented as the collaboration between the architect’s queries and ChatGPT’s responses.
  • B. Architectural Analysis: Architects query ChatGPT to formulate functional requirements, quality requirements, and relevant constraints for the campus bike application.The requested output covers functionality, quality, and constraints collectively referred to as ASRs.

C. Architectural Synthesis

Architectural synthesis transforms ASRs into UML-based models and refines those models with design patterns, tactics, and constraints through interaction with ChatGPT.

  • C. Architectural Synthesis: ASRs are synthesized into UML class and component diagrams representing the overall architecture and fine-grained design.Component diagrams represent the overall architecture, while class diagrams provide fine-grained representation.
  • C. Architectural Synthesis: Additional dialogue applies the singleton pattern, caching tactic, and data-minimization constraint to the class diagram.The singleton pattern restricts a single login session across devices, while caching and data minimization refine the design.
  • C. Architectural Synthesis: The architect uses ChatGPT to create a PlantUML script for the CampusBike class diagram.The script-generation instruction is part of the dialogue used to model the architecture.

D. Architectural Evaluation

Architectural evaluation checks whether the synthesized architecture satisfies ASRs and constraints using SAAM and individual or interacting scenarios. ChatGPT supports scenario generation and evaluation reporting for components such as ViewBike.

  • D. Architectural Evaluation: The synthesized architecture is evaluated against ASRs and constraints after modeling and refinement.Evaluation is used to assess whether the architecture satisfies the previously specified requirements.
  • D. Architectural Evaluation: ChatGPT presents individual scenarios for ViewBike and additional scenarios describing its interactions with other components.The scenarios support evaluation of both the component and its interactions.
  • D. Architectural Evaluation: An evaluation report is produced from the individual and interacting scenarios.The report is based on the interactions among the evaluated scenarios.
  • D. Architectural Evaluation: The evaluation process shares the functionality, quality, and constraints that have been evaluated.This forms part of the stated SAAM-oriented evaluation sequence.
  • D. Architectural Evaluation: The architect applies SAAM to evaluate the ViewBike component of the CampusBike application.The evaluation instruction explicitly names the Software Architecture Evaluation Method and the ViewBike component.

IV. RELATED WORK

Related work frames AI and software engineering research along two dimensions and motivates moving beyond traditional AI-based automation toward intelligence embedded in software engineering processes and solutions.

  • IV. RELATED WORK: The paper reviews prior research on AI in software engineering, architecture-centric software engineering, and ChatGPT in software development.These topics are identified as the relevant related-work areas.
  • IV. RELATED WORK: Research on synergizing AI and software engineering is classified as AI for SE and SE for AI.AI for SE concerns applying artificial intelligence in software engineering, while SE for AI concerns software engineering for artificial intelligence.
  • IV. RELATED WORK: The AI for SE perspective calls for methods that go beyond tool-based automation and pattern selection.The cited discussion emphasizes instilling intelligence in software engineering processes and solutions.

B. ChatGPT Assisted Software Engineering

ChatGPT is presented as an emerging DevBot for software engineering, with potential to support human-bot architecting while introducing consistency, ethical, intellectual-property, and bias concerns that require oversight.

  • Prior software-engineering use: ChatGPT can produce well-articulated programming solutions, but human oversight and intervention remain necessary.Earlier software-engineering studies reported reduced developer time and effort alongside the need for human supervision.
  • Research gap: The study identifies a gap in research on ChatGPT engaging software engineers in conversational dialogs to lead and support ACSE.It positions the work as complementary to research on software test automation and bug fixing with ChatGPT.
  • Socio-technical concerns: Collaborative architecting raises socio-technical concerns, including response variation, ethical and intellectual-property risks, and biased architectural recommendations.These concerns span inconsistent artifacts, possible privacy or regulatory violations, and recommendations influenced by adoption or training-data bias.
  • Human oversight: Iterative dialog and architect oversight can reduce response variation and help ensure generated architectural artifacts remain consistent and coherent.The paper specifically recommends refining outputs through repeated interaction and supervision.

B. Threats to the Validity

The study addresses validity threats through a systematic architecting process and scenario-based evaluation, but its single moderate-complexity case limits generalization and requires broader validation.

  • Future validation: Future work targets validity threats to improve methodological rigor and generalization of results.The paper frames validity threats as limitations, constraints, or potential flaws affecting generalization, replicability, and validity.
  • Internal validity: The case-study approach and incremental architecting helped analyze and refine the study, but other architecting processes and evaluation methods require investigation.This is identified as a remaining internal-validity concern.
  • External validity: A single case study of moderate complexity may compromise external validity across more complex settings and different levels of human expertise.The paper names cross-organisational development, mission-critical software, and novice or experienced engineers as contexts that may affect generalization.
  • Conclusion validity: Conclusions about architect productivity and ChatGPT efficacy require experimentation with multiple case studies, diverse teams, and real collaborative-architecting scenarios.The paper treats these additional studies as necessary for validating those conclusions.

VI. CONCLUSIONS AND FUTURE RESEARCH

The paper concludes that ChatGPT can participate in and empower human-led ACSE, while emphasizing response variance, ethical and socio-technical issues, and the need for empirical validation.

  • Conclusions: ChatGPT’s role as a DevBot for architecting software-intensive systems remains largely unexplored, motivating this case study.The study investigates both its potential and its perils in collaboration with a human architect.
  • Conclusions: The case study illustrates how ChatGPT can collaborate with a human architect to enrich existing ACSE processes with conversational assistance.The paper advocates augmenting tool-based AI automation with intelligence embedded in established processes.
  • Conclusions: Collaborative architecting must consider response and artifact variance, ethical implications, human supervision, and legal and socio-technical issues.These factors are identified as relevant when integrating ChatGPT into software engineering or ACSE processes.
  • Future research: Future research will study architect feedback and validation while integrating ChatGPT into software-service development for quantum computing systems.The motivation includes limited human expertise spanning software engineering and quantum physics.
Loading 2302.14600v1…