Source-linked AI summary
Exploring Challenges and Opportunities to Support Designers in Learning to Co-create with AI-based Manufacturing Design Tools
Frederic Gmeiner, Humphrey Yang, Lining Yao, Kenneth Holstein, Nikolas Martelaro
TL;DR
Designers face an underexplored learning challenge because AI co-creators operate differently from conventional CAD, and little is known about supporting co-creation on authentic design tasks. The paper observed designers using two AI manufacturing-design tools and found challenges in understanding and adjusting outputs, communicating goals, and collaborating with AI. It identifies strategies and design opportunities for supporting that learning.
Problem
Little is known about how to support designers learning to co-create with AI tools on authentic design tasks, despite their distinct role from conventional CAD.
Source: §1 · p. 1, §2.3 · p. 3Method
Results
Designers faced challenges understanding and adjusting AI outputs, communicating goals, and working collaboratively, while those who overcame challenges explored tool capabilities and reflected on design issues.
Source: §8 · p. 13Takeaways & Limitations
Support opportunities include scaffolding capability exploration and reflection, improving AI contextual awareness, and enabling more conversational multimodal communication.
Source: §8 · p. 13Participants were a subset of designers, mostly with minimal industry experience, and knew their designs would not be manufactured in a research study.
Source: §8.5 · p. 16
Abstract
from arXiv · showhide
AI-based design tools are proliferating in professional software to assist engineering and industrial designers in complex manufacturing and design tasks. These tools take on more agentic roles than traditional computer-aided design tools and are often portrayed as "co-creators." Yet, working effectively with such systems requires different skills than working with complex CAD tools alone. To date, we know little about how engineering designers learn to work with AI-based design tools. In this study, we observed trained designers as they learned to work with two AI-based tools on a realistic design task. We find that designers face many challenges in learning to effectively co-create with current systems, including challenges in understanding and adjusting AI outputs and in communicating their design goals. Based on our findings, we highlight several design opportunities to better support designer-AI co-creation.
1 INTRODUCTION
AI-enabled design tools can expand what designers create, but their more agentic, collaborative roles introduce a learning challenge distinct from conventional CAD. This study examines designers’ challenges, learning strategies, support needs, and expectations for co-creation.
1 INTRODUCTION
- Designers face a learning curve because AI co-creators operate differently from human collaborators and complex CAD tools.Unlike conventional CAD, designers formulate goals for AI rather than directly manipulating 3D geometry.Source: §1 · p. 1
- In think-aloud sessions with two AI tools, designers valued AI assistance but struggled to co-create effectively and interpret outputs.Those producing feasible, satisfying designs tested the tools’ boundaries, self-explained observed behavior, and sketched and reflected on design issues.Source: §1 · p. 2
- Human guides supported learning through step-by-step instructions, design reflection, alternative strategies and goals, and multimodal communication.Guides used screen annotations and mouse gestures to communicate with designers.Source: §1 · p. 2
- Participants often felt unable to communicate design goals to AI and wanted more conversational interactions and contextual awareness.These needs informed the paper’s discussion of support implications and future work.Source: §1 · p. 2
- The study contributes observed challenges, designers’ needs and expectations, and design opportunities for supporting co-creation in advanced manufacturing design.The contributions concern engineering and architectural designers learning to collaborate with AI on complex tasks.Source: §1 · p. 2
2 RELATED WORK
Prior work describes increasingly capable AI design tools, but offers limited empirical evidence about how designers learn to use them on realistic tasks. The paper draws on complex-software learning, human collaboration, and team learning to frame this gap.
2.1 AI-based design tools for manufacturing
- AI manufacturing tools generate designs from goals and requirements, but black-box interactions can make it difficult to develop a mental model of tool behavior.Few empirical studies examine how designers learn these tools on realistic tasks; this study provides observations of that process.Source: §2.1 · p. 2
2.2 Learning complex software
- Learning complex software is already demanding, while support for self-directed learning with active AI collaborators remains an open question.Prior research has proposed tutorials, feedforward tips, trial-and-error widgets, and explainability or intelligibility approaches.Source: §2.2 · pp. 2–3
2.3 Human-human collaboration as a lens for studying co-creative systems
- Human-collaboration concepts such as grounding and shared mental models inform AI collaboration, but authentic design-task learning remains underexplored.Design problems are often ill-defined and require teams to negotiate problem and solution spaces iteratively.Source: §2.3 · p. 3
2.4 Team learning
- Team learning studies how groups learn to collaborate, and this paper applies its concepts of communication, joint information processing, and action coordination to designer-AI co-creation.The authors note that team learning has been underused as a lens for human-AI co-creation.Source: §2.4 · p. 3
3 STUDY 1: THINK-ALOUD DESIGN SESSIONS
The first study used think-aloud sessions and interviews to observe designers new to AI as they completed realistic advanced-manufacturing tasks. Participants had relevant CAD experience but no experience with the studied AI tools.
3 STUDY 1: THINK-ALOUD DESIGN SESSIONS
- Trained designers new to AI completed realistic manufacturing design challenges in think-aloud sessions using one of two AI design tools.Mechanical engineering participants designed a light, strong engine bracket with Fusion360 Generative Design; architecture or industrial design participants designed a shape-changing-material bike bottle holder with SimuLearn.Source: §3 · p. 3
- After the task, participants submitted designs and joined semi-structured interviews; researchers collected session and interview recordings, transcripts, and designs.The recordings and transcripts included video and audio from open-ended design sessions and audio from post-task interviews.Source: §3 · p. 3
- Table 1 summarizes Study 1 participants; two listed IDs refer to the same person, and gaps reflect dropouts or assignment to Study 2.P-F11* and P-S11* were the same participant with experience using both Fusion360 and SimuLearn.Source: Table 1 · p. 4
3.1 Participants
- The study recruited 14 designers with at least two years of CAD experience and no experience with the studied AI tools.Participants came from architecture, industrial design, or mechanical engineering; most were students, and three had over 30 years of industry experience.Source: §3.1 · p. 4
AI-based design tools and tasks
The study paired two manufacturing design tasks with AI tools that differed in purpose, interaction style, and speed. Both tasks required iterative design, but one used a structured solver workflow while the other offered rapid simulation and multiple assistance levels.
AI-based design tools and tasks
- Participants tackled two realistic, complex tasks selected with engineering and manufacturing experts: mechanical optimization and shape-changing-material industrial design.Each task had an existing functional AI tool and was designed to fit work within a few hours over multiple sessions.Source: AI-based design tools and tasks · p. 4
- Fusion360 Generative Design generated lightweight, strong engine-bracket options from designer-specified requirements, which participants evaluated and could iterate by adjusting criteria.Participants began with geometric constraints, specified design criteria, and identified three viable designs for submission.Source: §3.2.1 · p. 4
- SimuLearn helped designers create shape-changing grids through simulation and optimization, for a bike bottle holder that had to fit around the bottle and frame.Designers modeled grid geometry and actuator ratios, observed predicted transformations, and could choose autonomous or interactive optimization.Source: §3.2.2 · p. 5, Figure 2 · p. 6
3.3 Choosing design tools
- The tools represented different purposes and interaction paradigms: Fusion360 structured parameter setup and could take hours to return solutions, while SimuLearn ran in 5–180 seconds.SimuLearn had no predefined workflow and let participants switch among three levels of AI support; Fusion360 users could export or rerun with adjusted parameters.Source: §3.3 · p. 5
4 STUDY DESIGN
Study 1 used staged think-aloud design sessions followed by interviews to observe designers learning the tools. The researchers assessed design outcomes and analyzed participants’ interactions and reflections.
4.1 Study procedure
- Participants received tool and task instructions, worked through an initial session and self-paced homework while thinking aloud, then completed a post-submission interview.The initial session was interrupted after 30 minutes; homework had no time limit.Source: §4.1 · p. 6, Figure 3 · p. 6
4.2 Analysis
- Researchers assessed satisfaction and completion time, checked engine brackets against shape and structural requirements, and relied on submission and self-reported satisfaction for bottle holders.Bracket structural soundness was checked with finite element analysis, while volume and shape requirements were also assessed.Source: §4.2.1 · p. 7, Table 2 · p. 7
- Video interaction analysis tracked task-relevant AI-feature use and whether actions produced satisfactory outcomes across the two tools.The analysis included bracket parameters and SimuLearn’s hybrid and automated optimization features.Source: §4.2.2 · p. 7
- Reflexive thematic analysis combined think-aloud and interview data through iterative coding, team theme development, and review of annotated video summaries.Researchers used semantic and latent coding and sought to distinguish usability issues from collaboration themes.Source: §4.2.3 · p. 7
5 RESULTS
Participants saw potential value in AI assistance, but most struggled to co-create effectively. They faced difficulties interpreting outputs, sharing control, and communicating goals, while some developed strategies to learn the tools.
5.1 Design Tasks Outcome Summary
- Only P-F05 produced an engine bracket meeting both shape and structural requirements; nearly all bottle-holder designers were dissatisfied or neutral, and one stopped without submitting.P-F01 met only the bracket’s shape requirements; all but one participant submitted a bottle-holder design.Source: §5.1 · p. 8, Figure 4 · p. 8, Figure 5 · p. 8
- In Fusion360, most designers learned to set structural constraints and obstacle geometry, but many mis-specified loads or the boundary box; in SimuLearn, all learned manual adjustment and simulation, while many avoided AI features.Only two designers used SimuLearn’s AI-assistance features successfully on the first try.Source: §5.1 · p. 8, Table 3 · p. 9, §5.2.2 · p. 10
5.2 Challenges designers faced when learning to co-create with AI (RQ1a)
- Designers struggled to interpret unusual AI outputs, identify their causes, and determine how to correct them, sometimes hesitating to alter optimized geometry.Some were unsure whether apparent flaws reflected design intent or algorithmic glitches.Source: §5.2.1 · pp. 8–9
- When tools seemed to dominate the process, designers accepted imperfect results, improvised workarounds, or abandoned AI assistance for manual work.Some SimuLearn users could not make sense of AI outputs even when they used the feature correctly.Source: §5.2.2 · pp. 9–10
- Designers struggled to communicate goals when they lacked knowledge of parameters, relied on defaults, or misunderstood how loads were applied, and few corrected such mistakes.The system applied the same load to each target rather than distributing it, producing higher loads and unnecessarily bulky brackets.Source: §5.2.3 · p. 10
5.3 Learning strategies among successful designers (RQ1b)
- Some designers deliberately tested parameter values and documented resulting changes to build mental models of the tools’ capabilities and limits.Others used sketches and verbal explanations to reason through forces, constraints, and gravity-related transformations.Source: §5.3.1 · p. 10
- Sketching and explaining design problems helped some designers recognize and overcome issues such as over-constraining a bracket or misjudging gravity’s influence.All designers initially tried trial and error and occasionally consulted help resources, but most still struggled to co-create effectively.Source: §5.3.2 · pp. 10–11, §5.3 · p. 10
6 STUDY 2: LEARNING WITH A PEER GUIDE (RQ2)
Study 2 paired designers new to AI co-creation with experienced peer guides during design tasks. The observed strategies included responding to uncertainty, prompting reflection, suggesting alternatives, and using visual communication.
6.1 Method
- Six additional designers worked with experienced Study 1 participants as guides; guided homework was limited to 50 minutes and followed by separate interviews.Guides were experienced with the tools, domain, and task, and received no script so their natural support strategies could be observed.Source: §6.1 · p. 11
6.2 Results – Guided Sessions
- Guides supported tool use and interpretation of AI behavior, capabilities, and limitations, typically responding to requests, uncertainty, or observed mistakes.They sometimes suggested alternative design goals or strategies.Source: §6.2 · p. 11
6.2.1 Guide
- Guides gave step-by-step parameter instructions and reacted to uncertainty by offering help or alternative strategies.They often noticed hedging language such as “I don’t know” or “I assume.”Source: §6.2.1 · p. 11, §6.2.2 · p. 11
- After simulations or solver runs, guides prompted designers to compare generated results with what they had envisioned.These questions prompted reflection and helped guides understand how to support the designer.Source: §6.2.3 · p. 12
- Guides suggested alternative means of expressing goals, such as extending obstacle geometry to reserve bolt clearance.The example helped the designer communicate the goal and develop an intuition for using the AI’s capabilities.Source: §6.2.4 · p. 12
- Screen annotations, sketches, and mouse gestures helped guides and designers clarify design strategies and develop shared mental models.Guides and designers used pointing and annotations to identify particular grid points and beams.Source: §6.2.5 · p. 12
7 DESIGNERS’ NEEDS AND EXPECTATIONS FOR CO-CREATING WITH AI-BASED DESIGN TOOLS (RQ3)
Across both studies, designers described needs that shaped what they expected from AI co-creation tools. They wanted more awareness of design context, reciprocal conversation, help reasoning through problems, and relevant examples.
7.1 Designers expect the AI system to have more contextual awareness about the design problem at hand
- Designers wanted tools to understand a part’s function and interfaces so they could proactively flag issues such as whether a bolt would fit.Some also wanted manufacturing and material suggestions or simulations of real-world use, such as bottle-holder friction.Source: §7.1 · p. 12
7.2 Designers desire a more conversational form of interaction with the tool
- Participants wanted reciprocal, conversational interaction that let them check whether designs met expectations for the part, loads, and materials.Some compared current interaction to programming or working with a skilled teammate who was not listening.Source: §7.2 · p. 12
7.3 Designers require support in thinking through design problems
- Designers valued peer guides’ help thinking through design problems and expected a co-creative tool to offer similar insight.One designer described the guide’s engineering perspective as revealing considerations they had not thought of.Source: §7.3 · pp. 12–13
7.4 Designers expect an AI tool to provide project-relevant work examples
- Designers wanted project-relevant examples from other users to help them understand tool capabilities and find creative inspiration.They described examples as useful even when the tool retrieved rather than generated the designs.Source: §7.4 · p. 13
8 DISCUSSION
Designers struggled to co-create with AI beyond learning interfaces: they had difficulty communicating goals, interpreting outputs, and coordinating with systems whose capabilities and context were opaque. The discussion proposes learning, reflection, contextual-awareness, and multimodal-communication supports, while bounding the findings to a particular participant group and study setting.
8.1 New learning challenges for human–AI co-creation
- Designers faced challenges understanding and adjusting AI outputs, communicating design goals, and collaborating with systems; these challenges went beyond interface learning and included planning parameters before iterative testing.Most participants did not produce feasible and satisfying outcomes despite CAD experience and interface resources, though some learned to co-create effectively.Source: §8 · p. 13, §8.1 · p. 13
8.2 Toward models of human collaboration as lenses for studying and designing co-creative systems
- Designers who systematically explored AI capabilities and limitations better predicted its actions and produced more satisfactory results, while many felt the tools were uncollaborative and took more control than desired.Guided sessions used uncertainty-responsive instructions and reflection prompts to help designers articulate intentions and coordinate next steps.Source: §8.2 · p. 13, Multi-modal interactive communication… · pp. 14–15
8.3.1 Scaffolding inductive learning of AI’s capabilities and limitations
- Hands-on parameter experiments, input-output examples, and project-related examples could help designers learn AI capabilities and limitations and build useful mental models.These proposed supports draw on observed inductive learning strategies and peer guides’ step-by-step instructions.Source: §8.3.1 · p. 15
8.3.2 Prompting designers’ planning and reflection
- Prompts for self-explanation and reflection, alternative goals, and generative questions could support planning, especially with tools whose long processing times limit rapid adjustment.The discussion notes that participants who did not reflect on design problems often failed to understand the system well enough to produce satisfactory outcomes.Source: §8.3.2 · p. 15
8.3.3 Improving co-creative AI tools’ contextual awareness of designers’ tasks and objectives
- Tools could build task context from design briefs or user behavior and respond to verbalized uncertainty or parameter-level uncertainty signals with reactive help.The discussion proposes tracking designers’ certainty and resurfacing parameters to investigate unexpected outcomes.Source: §8.3.3 · p. 15
8.3.4 Multi-modal
- Conversational clarification and negotiation, alongside sketching, annotations, or gestures over generated results, could help designers communicate goals and develop shared understandings with AI.The discussion also identifies multimodal communication as a support opportunity beyond manufacturing, including in image, music, and text-generation domains.Source: §8.3.4 · pp. 15–16, §8.4 · p. 16
8.4 Human–AI co-creation beyond manufacturing
- The study’s findings and support opportunities may apply beyond manufacturing to image, music, or text co-creation, but the participants represented only a subset of relevant designers and were mostly minimally experienced in industry.The authors also note that participants knew designs would not be manufactured, so professionals might have spent longer learning to produce feasible designs.Source: §8.4 · p. 16, §8.5 · p. 16
9 CONCLUSION
The paper reports an empirical study of how engineering and architectural designers learn to use AI-based manufacturing design tools. It documents co-creation challenges and needs, identifies support opportunities informed by human-human teamwork, and aims to encourage tools that combine human and AI strengths for complex design.
9 CONCLUSION
- The empirical study examined how engineering and architectural designers learn to work with AI-based manufacturing design tools.Source: §9 · p. 16
- The study documented designers’ challenges and needs and identified support opportunities informed by effective human-human teams.Source: §9 · p. 16
- The authors aim to inspire co-creative tools that combine human and AI strengths for complex designs neither could achieve alone.Source: §9 · p. 16
A ADDITIONAL MATERIALS
The additional materials include an overview of engine-mounting-bracket designs from the Fusion360 task and a table describing the post-task interview protocol.
A ADDITIONAL MATERIALS
- Figure 6 presents all engine mounting brackets created in the Fusion360 task.Source: Figure 6 · p. 19
- Each participant submitted three design options, with each trio generated from the same parameter values.Source: Figure 6 · p. 19
- Table 6 contains the semi-structured post-task interview questions.Source: Table 6 · p. 20