Source-linked AI summary
CreativeConnect: Supporting Reference Recombination for Graphic Design Ideation with Generative AI
DaEun Choi, Sumin Hong, Jeongeon Park, John Joon Young Chung, Juho Kim
TL;DR
Graphic designers often need to recombine references for ideation, but discovering useful elements and exploring combinations is effortful and generative-AI support for expanding idea variety remains underexplored. CreativeConnect uses generative-AI pipelines to extract and expand keywords and present diverse recombinations as sketches with descriptions; users generated more ideas and perceived greater creativity than with a baseline.
Problem
Reference recombination is effortful, and generative-AI interaction for expanding ideation variety through recombined references remains underexplored.
Method
CreativeConnect extracts reference-image keywords, recommends related keywords, and generates diverse keyword recombinations as sketch-and-description pairs.
Results
Participants using CreativeConnect produced more design ideas and perceived their sketches as more creative than with the baseline system.
Takeaways & Limitations
CreativeConnect supported discovering reference elements and generating recombined design ideas, with discussion of extending it to other creativity-support tasks.
Takeaways & Limitations
The 30-minute study conditions were shorter than real design processes, limiting observation of behavior over longer periods.
Abstract
from arXiv · showhide
Graphic designers often get inspiration through the recombination of references. Our formative study (N=6) reveals that graphic designers focus on conceptual keywords during this process, and want support for discovering the keywords, expanding them, and exploring diverse recombination options of them, while still having room for designers' creativity. We propose CreativeConnect, a system with generative AI pipelines that helps users discover useful elements from the reference image using keywords, recommends relevant keywords, generates diverse recombination options with user-selected keywords, and shows recombinations as sketches with text descriptions. Our user study (N=16) showed that CreativeConnect helped users discover keywords from the reference and generate multiple ideas based on them, ultimately helping users produce more design ideas with higher self-reported creativity compared to the baseline system without generative pipelines. While CreativeConnect was shown effective in ideation, we discussed how CreativeConnect can be extended to support other types of tasks in creativity support.
1 INTRODUCTION
CreativeConnect addresses the effort and uncertainty of recombining reference elements during graphic-design ideation. It uses generative AI pipelines to extract and expand keywords, generate varied recombinations, and present them as incomplete sketches with descriptions, improving idea generation and perceived creativity over a baseline.
- Motivation: Graphic designers spend substantial time dissecting references, evaluating combinations, and sketching alternatives, especially when identifying and integrating disparate elements is difficult.The authors note that this effort is particularly challenging for less experienced designers.
- Formative Findings: The formative study identified conceptual ideation as the main recombination stage, with designers focusing on subject matter, action and pose, theme and mood, and composition.Visual development instead emphasizes adding details such as color and texture after a direction has been chosen.
- Design Goals: The proposed design goals are effortless element specification, relevant-element recommendation, numerous recombinations, and intentionally incomplete outputs that preserve designers’ creative input.These goals respond to designers’ concern that exhaustive manual exploration is impractical and overly finished support could diminish their contribution.
- System: CreativeConnect extracts keywords from reference images, recommends related keywords, and generates recombination options as sketch-and-description pairs.Its generative pipelines automate keyword extraction, recombination generation, and transformation into textual and visual outputs.
- Evaluation: Participants using CreativeConnect produced more design ideas in a fixed time and perceived their sketches as more creative than with the baseline system.The study also found support for both discovering reference elements and generating ideas by recombining them.
2 RELATED WORK
Prior work supports reference collection, diversity, decomposition, recombination, and generative-AI collaboration, but has focused less on expanding idea variety through reference recombination during ideation. CreativeConnect builds on these strands by combining reference organization with AI-supported keyword recombination.
- References in Ideation: References support design ideation by helping designers understand problem spaces, use analogical thinking, explore diverse ideas, and organize inspiration through mood boards.Diverse references can help prevent fixation, while mood boards support comprehension and interpretation of ephemeral design elements.
- Recombination: Combinatorial creativity forms new ideas by recognizing differences among existing concepts and blending them, with diversity supporting novel associations.Computational recombination systems have been applied in domains including chair design and text-based ideation.
- Recombination: Genetic exploration and related systems merge elements from existing designs across garden design, 3D modeling, architecture, and 2D graphics.These approaches primarily enrich references during information gathering rather than directly supporting the paper’s ideation focus.
- Prior Systems: Reference-decomposition tools expose fine-grained semantic, color, shape, or structural aspects, while blending systems combine objects, icons, images, styles, or fashion concepts.Examples include CollageMachine, MetaMap, VisiBlends, VisiFit, ICONATE, PopBlends, FashionQ, and Artinter.
- Generative AI: Generative-AI research has enabled text-, layout-, sound-, and style-conditioned image generation, iterative prompt refinement, prompt mixing, and human-AI co-creation.These approaches primarily address expressing intentions accurately or collaborating during design execution.
- Research Gap: The remaining gap is how to design generative-AI interaction that inspires graphic designers by recombining references to expand ideation variety.The paper positions CreativeConnect as an AI-infused creativity-support tool addressing this gap.
3 FORMATIVE STUDY
The formative study examined how early-stage designers recombine references and found that recombination centers on conceptual ideation. Designers needed help discovering overlooked and related elements, exploring many combinations, and retaining room for their own creativity.
- Study Design: Six early-stage designers were studied through reference-search observation, sketching tasks, and semi-structured interviews, with coded findings reaching saturation after six sessions.Participants included designers with limited professional experience and design students.
- 3.3.1 Early-Stage Design Ideation Focuses on Conceptual Aspects.: Participants distinguished conceptual ideation from visual development, using recombination primarily during the initial conceptual stage rather than when adding later visual details.Conceptual ideation focused on conveying the topic, whereas visual development focused on color, texture, and completion.
- 3.3.2 Types of Elements Used for Recombination: Designers extracted reference elements including objects, actions and poses, themes and moods, and compositional arrangements.Objects were a common direct input to sketches, such as using a paper plane to convey playfulness and tour service.
- 3.3.2 Types of Elements Used for Recombination: Elements that initially appeared appealing were not always those eventually used, because closer examination revealed new elements of interest.This motivated support for helping designers discover overlooked reference elements.
- 3.3.3 Challenges During Finding Elements: Designers generated new keywords from discovered elements, but expanding keywords was difficult and was seen as a promising opportunity for system recommendations.One example combined “toy blocks” and “train” into the new keyword “toy train.”
- 3.3.4 Challenges During Recombining Elements: Three of six participants worried about missing better combinations, while four relied on mental imagination because sketching every possibility was too time-consuming.The findings motivated systems that generate diverse recombination options and reduce uncertainty about unexplored combinations.
- 3.3.5 System Support should be Incomplete: Participants deliberately excluded visual details during conceptual recombination, preferring incomplete sketches that leave room for their own creativity.The resulting design goal was to present recombinations in an intentionally incomplete format rather than as highly detailed artwork.
4 CREATIVECONNECT
CreativeConnect supports early-stage conceptual ideation by extracting and recommending keywords from references, then generating diverse sketch-based recombinations while preserving room for user reinterpretation.
- System Overview: CreativeConnect extracts four keyword types—subject matter, action & pose, theme & mood, and arrangement—from reference images for user selection.Captioning and language models identify the first three categories, while segmentation detects arrangement and composition.
- System Overview: Users organize references and selected keywords on a mood board, receive additional keyword recommendations, and choose combinations for ideation.Recommendations are based on keywords added to the board or on specific user selections.
- Generation Pipeline: CreativeConnect presents recombination options as sketch images with one-line descriptions so designers can further reinterpret partially unfinished outputs.The system intentionally keeps generated outputs in line-sketch form to support designers’ own creative input.
- User Scenario: In the scenario, three distinct drafts combined selected Christmas and underwater keywords in ways the designer found difficult to develop independently.The designer could request five additional sketches using the same description after finding one concept interesting but disliking its initial sketch.
- Generation Pipeline: The recombination pipeline generates three textual descriptions from selected keywords, varies the chosen arrangement, and renders each option as an image transformed into a sketch.The pipeline uses language models for descriptions, a layout variator for arrangements, an image generator, and style transfer for sketches.
- Pipeline Evaluation: Keyword recommendations were moderately similar to original keywords, falling between synonym and irrelevant groups in similarity.The reported similarities were 0.774 for synonyms, 0.696 for recommendations, and 0.624 for irrelevant keywords.
5 EVALUATION
The evaluation compared CreativeConnect with a baseline in a within-subjects study of 16 design participants, examining reference-element discovery, recombination, ideation outcomes, and user experience.
- Study Design: The within-subjects study with 16 participants evaluated CreativeConnect’s support for finding reference elements and recombining them into ideas.The evaluation also examined idea quality and quantity and how participants used system outputs during ideation.
- Study Design: The baseline retained a similar interface but omitted keyword extraction, keyword recommendation, and automated recombination options.Participants instead added keyword notes manually, specified layouts and prompts, and used ChatGPT for support.
- Study Procedure: The two-hour procedure used two counterbalanced tool-and-task sessions, each with a 30-minute ideation phase and post-task survey.Participants received tutorials before each round, could sketch ideas on paper, and had a break between rounds.
- Measures: The evaluation measured perceived usefulness across ideation steps, satisfaction with sketch outcomes, behavioral logs, and expert ratings of creativity and diversity.Usage logs captured sketch timing and generated images, while two art experts rated 96 sketches on 7-point scales.
6 RESULTS
CreativeConnect supported both keyword discovery and recombination, helping participants explore more diverse inputs, generate more sketches within a fixed session, and perceive greater creativity and usefulness than the baseline on several measures.
- Finding Keywords from the Reference: CreativeConnect significantly increased perceived help in discovering valuable reference elements (M=6.13 vs. 3.75; p=0.001).The systems did not differ significantly in perceived help for organizing references.
- Finding Keywords from the Reference: Participants added significantly more keyword notes with CreativeConnect than with the baseline (M=34.69 vs. 13.19; p<0.0001).CreativeConnect users continued extracting keywords throughout the process, whereas baseline users mostly extracted them during the initial sketch.
- Recombining Elements: CreativeConnect was significantly more helpful for generating multiple ideas from collected elements (M=5.94 vs. 4.88; p=0.023).Its keyword inputs also had significantly lower minimum semantic similarity than baseline inputs (M=0.222 vs. 0.263; p=0.008), indicating more diverse recombinations.
- Creativity and Diversity of the Final Sketches: Participants rated their sketches as significantly more creative with CreativeConnect (M=5.38 vs. 4.19; p=0.004), although expert creativity ratings were not significantly different.Twelve of 16 participants reported feeling more creative with CreativeConnect, especially when struggling to generate early-stage ideas.
- Efficiency of the Ideation Process: Participants produced significantly more sketches in 30 minutes with CreativeConnect than with the baseline (M=5.56 vs. 5.06; p=0.041).The additional keyword specification did not significantly increase perceived workload.
- Source of the Inspiration: CreativeConnect users drew on generated images, descriptions, reference and recommended keywords, often reinterpreting outputs rather than accepting them directly.Participants rated CreativeConnect more useful for producing favorite ideas (M=5.63 vs. 4.56; p=0.045), while baseline ChatGPT inspiration was limited by prompt and visual-task difficulties.
7 DISCUSSION
CreativeConnect supports reference recombination by stimulating exploration through keywords, recommendations, and incomplete outputs, complementing baseline support for intentional implementation. Its benefits vary with users’ needs, expertise, and design context.
- CreativeConnect’s support: CreativeConnect supports recombination by helping users extract elements, receive keyword recommendations, and explore diverse sketch-based combinations.The system presents recombination options as sketch images with one-line descriptions.
- Inspiration sources: CreativeConnect users were more inspired by the overall image concept, while baseline users focused on specific compositions or object details.Both conditions inspired users, but the level of abstraction differed.
- Two types of creativity support: CreativeConnect stimulates inspiration through keywords and partial expression, whereas the baseline emphasizes transparency, control, and faithfully implementing detailed input.CreativeConnect supports emergence, mutation, and combination through extraction, recommendation, and keyword merging.
- Low-fidelity output: 12 out of 16 participants preferred sketch output over complete images because low fidelity left room for imagination and interpretation.The authors suggest low-fidelity output can prevent fixation and facilitate creativity during ideation.
- Other contexts: CreativeConnect may support collaboration by preserving sequential keyword-noting and merging processes on the mood board and merging panel.Participants reported that it would be significantly helpful for collaborating with other designers.
- Other contexts: Applying CreativeConnect to other design domains requires identifying domain-specific reference elements and using those categories as pipeline keywords.The current goals and features are tailored primarily to illustration, which centers on visual subject matter.
8 LIMITATIONS AND FUTURE WORK
The authors identify limits in study duration, recombination methods, and AI familiarity, motivating longer real-world studies and broader pipeline support.
- Study scope: 30-minute ideation tasks were shorter than actual design processes, limiting observation of behavior over extended periods.Future work should incorporate CreativeConnect into real-world design projects and compare behavior with lab studies.
- Recombination methods: The pipeline combines selected keywords in image descriptions, but other recombination methods include blending objects or expressing keywords through visual details.Future work could investigate support for these alternative methods.
- AI familiarity: Results may depend on users’ familiarity with AI, a dimension not explored in the study.Future research can examine how AI knowledge and prior experience affect creativity-support outcomes.
9 CONCLUSION
CreativeConnect supports reference recombination by helping designers identify elements, receive keyword recommendations, and explore diverse options. Its low-fidelity sketches supported further imagination, while the study found more ideas and higher perceived creativity than the baseline.
- Contribution: CreativeConnect helps designers identify reference elements, receive relevant keyword recommendations, and generate diverse recombination options.The system is designed to support reference recombination and novel design-idea generation.
- Contribution: Low-fidelity sketch output encouraged imaginative exploration by leaving room for users to extend the system’s suggestions.The conclusion presents this as a notable aspect of CreativeConnect’s creativity support.
- Results: The user study found that CreativeConnect helped participants produce more design ideas and perceive their ideas as more creative than with the baseline.The reported benefits covered both finding and recombining reference elements.
A TECHNICAL DETAILS
CreativeConnect’s technical pipeline uses language-model prompts to extract categorized keywords, generate novel keyword recommendations, produce distinct text descriptions, and arrange objects spatially with bounding boxes.
- Keyword extraction: The extraction prompt categorizes illustration content into subject matter, action and pose, and theme and mood.Subject matter describes physical objects or characters; actions describe performed activities; themes describe overall mood or meaning.
- Keyword recommendation: The recommendation prompt semantically combines reference keywords to generate additional subject-matter, action-and-pose, and theme-and-mood terms.Examples show recommendations that expand the input into new concepts such as adventurous, serene, or joyful.
- Text recombination: The recombination prompt generates three significantly different illustration descriptions from user-selected keywords.Each description includes a scene and object descriptions that users can draw from.
- Layout matching: The bounding-box matcher assigns provided boxes to named objects so generated illustrations can be spatially balanced and realistic.It matches captioned objects with proportional top-left coordinates, widths, and heights.
- Layout generation: The bounding-box generator creates object boxes from an illustration caption and object list within 512x512 image boundaries.The prompt specifies top-left coordinates and prevents boxes from extending beyond the image.
A.6 Example Outputs from the Technical Pipeline
CreativeConnect’s technical pipelines include element extraction, keyword recommendation, and recombination generation. Together, these pipelines transform reference images and selected keywords into varied ideation inputs and outputs.
- CreativeConnect uses an element extraction pipeline to identify elements from reference images.
- A keyword recommendation pipeline expands the elements available for ideation.
- A recombination generation pipeline produces outputs by combining selected elements.
B USER STUDY
The baseline system provides comparable mood-board interactions but requires users to enter keywords, configure layouts, and write prompts manually. It lacks CreativeConnect’s automated keyword extraction and recommendation features.
- The baseline system lets participants add keyword notes manually instead of extracting keywords automatically.
- The baseline mood board supports the same core interactions as CreativeConnect but has no keyword suggestion panel.
- Users manually configure layouts and image-generation prompts rather than selecting keywords to combine.
B.2 Interview questions
The interview examined how participants experienced baseline and CreativeConnect during idea generation. Questions addressed tool differences, stage-specific usefulness, output format, and how generated images entered participants’ design processes.
- Comparative experience: Participants were asked to compare the baseline and CreativeConnect idea-generation processes and their differences from usual practice.
- Stage-specific usefulness: The interview assessed which tool helped more with finding reference elements, exploring ideas, and generating sketches.
- Output use: Participants discussed whether sketch-format outputs were useful and how they incorporated generated images into their designs.
B.3 Additional User Study Results: Raw Usage Log
The raw usage log records timestamps for keyword additions, image-generation inputs, and completed sketches across the study. Because one participant’s memo-writing was not timestamped, that participant’s data were excluded from action-to-sketch-turn analysis.
- The usage log records three action types: adding keyword notes, generating images, and completing design-idea sketches.
- P15 generated ideas before sketching them collectively later in the session, so sketch timestamps did not identify each idea’s preceding actions.
- P15’s usage data were excluded from analysis linking action types to individual sketching turns.