PaperlayerView exact revision on arXiv

Source-linked AI summary

How Generative AI supports human in conceptual design

Liuging Chen, Yaxuan Song, Jia Guo, Lingyun Sun, Peter Childs, Yuan Yin

arXiv:2502.00283v1cs.HC

TL;DR

Empirical evidence about how Generative AI affects different stages of conceptual design remains limited. This study tested text-to-text and text-to-image assistance with novice designers and found that AI mainly supported problem definition and idea generation, while selection and evaluation remained human-led and were enhanced with assistance.

  • Problem

    Evidence about Generative AI’s effects across conceptual-design stages remains limited, despite the use of design-support methods that still require designers’ cross-domain reasoning.

    Source: §2.1 · p. 3, §2.2 · p. 4

  • Method

    The study examined 20 novice designers completing two conceptual-design tasks with or without GPT-3.5, Midjourney, or both.

    Source: §1 · p. 2, §3 · p. 4, §6 · p. 16

  • Results

    Generative AI mainly assisted problem definition and idea generation, while selection and evaluation remained predominantly human-led and were enhanced with assistance.

    Source: §1 · p. 2, §6 · p. 16

  • Takeaways & Limitations

    The findings provide stage-level evidence about Generative AI’s role in conceptual design and inform implications for future design support.

    Source: §1 · p. 2

    The experiment focused on novice designers, and the Combined Group consistently used ChatGPT before Midjourney, leaving tool-use order and findings across more experienced designers for further investigation.

    Source: §5.3 · p. 16

Abstract

from arXiv · show

Generative Artificial Intelligence (Generative AI) is a collection of AI technologies that can generate new information such as texts and images. With its strong capabilities, Generative AI has been actively studied in creative design processes. However, limited studies have explored the roles of humans and Generative AI in conceptual design processes, leaving a gap for human-AI collaboration investigation. To address this gap, this study uncovers the contributions of different Generative AI technologies in assisting humans in the conceptual design process. Novice designers completed two design tasks with or without the assistance of Generative AI. Results revealed that Generative AI primarily assists humans in problem definition and idea generation stages, while idea selection and evaluation remain predominantly human-led. Additionally, with Generative AI assistance, the idea selection and evaluation stages were further enhanced. Based on the findings, we discuss the role of Generative AI in human-AI collaboration and implications for enhancing future conceptual design support with Generative AI assistance.

1. Introduction

Conceptual design challenges novices because existing methods and tools still depend on designers’ knowledge and reasoning. This study examines how Generative AI supports different stages of conceptual design, finding that it mainly assists problem definition and idea generation while selection and evaluation remain human-led and are enhanced with AI assistance.

1. Introduction

  • Existing design theories structure conceptual design, but their effective application still depends on designers’ knowledge and experience, challenging novices.Semantic networks and case databases provide stimuli but not situation-specific solution suggestions, leaving designers to reason from the case domain to the problem domain.Source: §1 · pp. 1–2
  • Text-to-text and text-to-image Generative AI have been studied for guiding design, supporting divergent and convergent thinking, generating solutions, and visualizing ideas.Text-to-image tools can also generate visual stimuli from user prompts and reduce the time and skill demands of manual sketching.Source: §1 · p. 2
  • Generative AI primarily assists problem definition and idea generation, while idea selection and evaluation remain predominantly human-led and are further enhanced with AI assistance.The study examines two design tasks completed with or without ChatGPT or Midjourney assistance.Source: §1 · p. 2
  • The study addresses limited empirical evidence about Generative AI’s effects across conceptual-design stages by examining its support for humans in the process.The researchers assessed stages of assistance, human-led stages, tool performance, design outputs, and participant prompt strategies.Source: §1 · p. 2
  • The study contributes empirical, stage-level evidence about Generative AI’s role in conceptual design and discusses implications for future design support.Source: §1 · p. 2

2. Literature review

The literature review describes conceptual design as a key but demanding phase and surveys design support methods and Generative AI tools. It identifies limited empirical evidence about Generative AI’s effects across conceptual-design stages and motivates an experiment focused on novice designers.

2.1. Conceptual design

  • The study defines conceptual design as four stages: problem definition, idea generation, idea selection and evaluation, and idea evolution.This framework draws on earlier conceptual-design models and underpins the experiment and its conclusions.Source: §2.1 · p. 3
  • Stimulus-based support methods offer knowledge or design stimuli, but designers must still use cross-domain reasoning to adapt concepts to the current problem.The review notes that obtaining original and novel ideas through designers’ own effort is challenging.Source: §2.1 · p. 3

2.2. Generative AI in conceptual design

  • Text-to-text and text-to-image models have attracted attention in conceptual design, including research on integrating them into existing workflows.Source: §2.2 · p. 3
  • Text-to-text tools such as ChatGPT can generate contextual natural-language responses and have been applied to requirements extraction, creative ideation, and solution generation.The experiment used GPT-3.5, selected for its coherent, contextually relevant text generation capabilities.Source: §2.2 · p. 4
  • Text-to-image models can provide visual references and multimodal stimuli for early concept development and support rapid visualization and iteration.The review identifies Midjourney as widely adopted in collaboration research for image quality and user-friendly features.Source: §2.2 · p. 4
  • The study responds to limited empirical evidence on Generative AI’s effects in conceptual design by examining text-to-text and text-to-image models across stages.Source: §2.2 · p. 4
  • The study focuses on novice designers because Generative AI may expand their exploration scope and lower barriers to cross-disciplinary design and rapid visualization.Source: §2.2 · p. 4

3. Experimental study

The experimental study uses general-purpose text-to-text and text-to-image Generative AI to investigate human-AI collaboration in conceptual design. It is organized around questions about stage involvement, AI performance, and prompt content.

3. Experimental study

  • The study selected Midjourney and GPT-3.5 as example text-to-image and text-to-text tools to investigate Generative AI’s role in conceptual design.The researchers chose general-purpose tools rather than design-specific alternatives to study Generative AI’s role, not to evaluate existing design-specific tools.Source: §3 · p. 4
  • The experiment asks which conceptual-design stages involve Generative AI, how the tools perform, and what characterizes participants’ prompts.Source: §3 · p. 5
  • Figure 1 represents the experimental study procedure.Source: Figure 1 · p. 5

3.1. Participants

The study recruited 20 novice designers who had prior experience with ChatGPT and Midjourney. Participants completed the experiment in about 70 minutes on average.

3.1. Participants

  • The study recruited novice designers with less than four years of design-learning experience and prior experience using ChatGPT and Midjourney.Prior tool experience was required so participants could focus on the design tasks rather than learning the tools.Source: §3.1 · p. 5
  • Twenty participants aged 18–26 took part: 13 females and seven males, with no restrictions on major background.Their mean age was 18–26 years, SD=2.7.Source: §3.1 · p. 5
  • Participants received $10 per hour and took about 70 minutes on average to complete the study.Source: §3.1 · p. 5

3.2. Procedure

Twenty participants were randomly assigned to four groups and completed two 20-minute conceptual design tasks after a brief training. The study collected design outputs, questionnaire responses, and post-task interviews.

3.2. Procedure

  • After a 20-minute training, participants completed two 20-minute design tasks: a baby chair and tangible music bricks.The tasks were selected to allow ideas to be conveyed through shape and external structure while leaving room for creative divergence.Source: §3.2 · pp. 5–6
  • Participants were randomly assigned to ChatGPT, Midjourney, Combined, or Human groups, with the Combined Group free to choose tool order.The Human Group worked without Generative AI; each AI group used its assigned tool or tools for both tasks.Source: §3.2 · pp. 5–6
  • Each participant produced one design image and essential text description for each task.Source: §3.2 · p. 6
  • After the tasks, participants completed a questionnaire and a roughly 30-minute semistructured interview about AI performance and collaboration.Human Group participants rated their own performance as a baseline; interviews discussed questionnaire results and participants’ evaluations and suggestions.Source: §3.2 · p. 6

3.3. Data collection and analysis

The study gathered participant assessments, expert ratings, reports of AI-helping and human-led stages, prompt data, and post-experiment interviews. Researchers used statistical testing, independent coding, and affinity diagramming to analyze these materials.

3.3.1. Participants’ assessment of Generative AI tools performances

  • Participants rated tool performance on six criteria, while Combined Group participants rated ChatGPT and Midjourney separately and Human Group participants rated their own performance.The six criteria were speed, subject, diversity, novelty, triggering more ideas, and requirement satisfaction; Human Group ratings served as a baseline.Source: §3.3.1 · pp. 6–7, Table 1 · p. 7

3.3.2. Expert ratings

  • Five professional designers rated 40 solutions on five criteria using a 7-point Likert scale.The criteria were novelty, feasibility, usability, functional diversity, and cost; a higher cost score meant poorer performance.Source: §3.3.2 · pp. 7–8

3.3.3. Generative AI’s helpful stages in conceptual design

  • A post-experiment questionnaire asked participants which stages AI helped with and which stages humans led, allowing multiple selections.“Actual-helping stages” were stages participants reported completing with AI assistance; human-led stages meant participants completed most of the work independently.Source: §3.3.3 · p. 8

3.3.4. Participants’ prompt

  • Researchers categorized 114 prompts by conceptual-design stage, then independently coded and reconciled assistance strategies through affinity diagramming.The prompt coding reached κ = 0.74; two researchers used sticky notes and iterative labels to group prompt contexts into strategies.Source: §3.3.4 · p. 8

3.3.5. Post-experiment interview

  • One-to-one post-task interviews were tailored to questionnaire responses and completed designs, probing tool use, ratings, human-led stages, collaboration, and participants’ broader views.Interviewers asked why and how participants used tools at particular stages and why they considered stages human-led or collaborative.Source: §3.3.5 · pp. 8–9

4. Results

The results describe where participants used Generative AI, how participants and experts assessed the tools and designs, and the strategies reflected in prompts across four conceptual-design stages. AI assistance was concentrated in early stages, while human judgment remained prominent in selection and evaluation.

4.1. RQ1: In which stages is Generative AI involved in?

  • Among 15 AI-assisted participants, AI help was reported most often for idea generation (86.7%) and problem definition (73.3%), while idea selection and evaluation was most often human-led (86.7%).Idea evolution was reported as both AI-helped and human-led by 60% of participants.Source: §4.1 · p. 9, Figure 2 · p. 10

4.2. RQ2: What are the performances of Generative AI?

  • Expert ratings gave Midjourney the highest overall score (mean 4.34, SD 0.75); ChatGPT and Midjourney groups both outperformed the Human Group overall.The overall differences versus Human were significant for ChatGPT (p = 0.02) and Midjourney (p = 0.01).Source: §4.2 · pp. 10–11, Table 3 · p. 11

4.3.1. Problem definition stage:

  • In problem definition, ChatGPT supported audience, user-needs, product, functionality, and materials research, while Midjourney helped explore visual design intents.Source: §4.3.1 · p. 12

4.3.2. Idea generation stage:

  • For idea generation, ChatGPT synthesized specified design points or generated ideas without specific directions, while Midjourney quickly visualized ideas and reduced manual sketching effort.Midjourney had its highest prompt count in this stage: 20/50 = 40.0%.Source: §4.3.2 · p. 12

4.3.3. Idea selection and evaluation stage:

  • In selection and evaluation, ChatGPT offered creativity and feasibility assessments, while Midjourney generated visual variants for comparing and selecting concepts.Source: §4.3.3 · pp. 12–13

4.3.4. Idea evolution stage:

  • During idea evolution, ChatGPT refined design elements and supported concept iteration, while Midjourney visualized revised concepts and added visual detail.Examples included improving comfort through material refinement and shifting a rocking-chair concept toward modular components.Source: §4.3.4 · p. 13

5. Discussion

Generative AI broadened novice designers’ solution exploration and was associated with higher solution-quality scores, while also prompting more idea selection and evaluation. The discussion distinguishes the roles of text and image models and identifies workflow, scope, and evaluation needs for future support.

5.1. The role of Generative AI in human-AI collaboration in conceptual design

  • AI-assisted groups scored higher on diversity, novelty, idea triggering, and all five solution-quality metrics; ChatGPT and Midjourney scores differed significantly from the Human Group.The authors note that focusing on triggering more ideas may leave potential design fixation unexplored.Source: §5.1 · pp. 13–14
  • Generative AI assistance made novices’ idea selection and evaluation more pronounced, as they reviewed and assessed generated outputs.Without AI, some participants described designing directly from existing ideas and overlooking selection and evaluation.Source: §5.1 · p. 14
  • ChatGPT helped novices define problems by outlining design elements, while Midjourney’s problem-definition help depended on solution-oriented prompts and a preliminary design direction.The discussion describes Midjourney as insufficient on its own for problem definition.Source: §5.1 · p. 14
  • ChatGPT could analyze design elements but leave novices without an integrated solution overview, whereas Midjourney visualized shape, texture, and color.The text notes that integrated solutions from filtered design points may confuse novice designers during idea generation.Source: §5.1 · pp. 14–15

5.2. Implications for future conceptual design support under Generative AI’s help

  • Combining ChatGPT and Midjourney produced no synergistic effect in participant assessments or expert ratings, and differing command interpretations could frustrate users.The authors recommend workflow guidance and system integration that match each model’s strengths to conceptual-design stages.Source: §5.2 · p. 15
  • Midjourney received the highest novelty feedback and higher expert-rated mean scores than ChatGPT, motivating further exploration of image stimuli and links between text output and visual solutions.The authors suggest visual search as one way to relate language-model responses to possible design solutions.Source: §5.2 · p. 15

5.3. Limitations and future directions

  • The study used text input with text and image outputs; future work could add voice, video, or sketches and test multimodal inputs and more reasoning-intensive design tasks.The authors note prior research finding that LLM output accuracy decreases as task complexity increases.Source: §5.3 · pp. 15–16
  • The comparisons used general-purpose AI versus solo design and a Human Group control; future studies could compare specialized or traditional tools, test usage order, and include experienced designers.Combined Group participants consistently used ChatGPT before Midjourney, despite having no order restriction.Source: §5.3 · p. 16

6. Conclusion

In an experiment with 20 novice designers, Generative AI mainly supported problem definition and concept generation, while selection and evaluation remained predominantly human-led. AI assistance improved participant feedback and expert ratings, but combining text-to-text and text-to-image models showed no synergistic effect.

6. Conclusion

  • Generative AI mainly supported problem definition and concept generation, while idea selection and evaluation remained predominantly human-led.The study involved 20 novice designers using text-to-text and text-to-image models, with or without assistance.Source: §6 · p. 16
  • AI assistance improved participants’ feedback and expert ratings, but combining text-to-text and text-to-image models showed no synergistic effect.Source: §6 · p. 17
  • The authors discuss Generative AI’s role in human-AI collaboration, compare model efficacy, and propose implications for more effective and user-friendly conceptual design support.Source: §6 · p. 17
Loading 2502.00283v1…