Source-linked AI summary
Exploring Normativity in Stable Diffusion: Insights for XAI in the Arts
Michelle Dutoit, Baptiste Caramiaux
TL;DR
Normative processes in generative systems remain underexplored, particularly regarding their effects on creative practice. The paper examines system deviations and practitioners’ responses, reporting disempowerment and simplified visual representation while proposing prompt transparency and related XAIxArts directions.
Problem
Normative processes in generative systems remain underexplored, including how their outputs can limit creative ideation and distort representations.
Method
The paper considers generated captions as a potential approach for highlighting discrepancies between input prompts and outputs.
Results
Participants perceived normative behavior through system deviations, including the unsolicited removal or addition of details, and reported frustration and powerlessness alongside simplified visual representation.
Takeaways & Limitations
The findings support prompt transparency and XAI as promising directions for going beyond default behavior in creative contexts.
Takeaways & Limitations
The discussed approaches introduce new challenges, and normative processes remain insufficiently understood.
Abstract
from arXiv · showhide
Generative text-to-image (T2I) systems are increasingly adopted in creative practice, yet their normative behaviors remain underexplored from the perspective of creative practitioners. In this workshop paper, we present a within-subject study with 14 creative practitioners using Stable Diffusion to create illustration from two tasks of differing specificity. We investigate whether and how practitioners perceive normative behavior in a T2I system and how it impacts their creative process depending on task specificity. Our findings show that participants perceived normative behavior through invariant patterns, stereotypical output, and unsolicited omission or addition of details. These experiences led to feelings of disempowerment and creative compromise. We discuss implications for XAIxArts, including prompt transparency and artist empowerment in creative contexts.
1 A Brief Background and Motivation
The paper frames T2I normativity as an underexplored issue in creative practice, where systems can reproduce cultural defaults and constrain artistic expression. It asks how creative practitioners perceive this behavior and how task specificity shapes its effects.
- T2I systems can misrepresent cultures, reinforce Western-centric norms, and produce stereotypical or inaccurate representations.
- The paper argues that understanding normativity could help XAIxArts address opaque normative bases and reduce their negative impacts on users.
- Prior work examines conveyed values and user workarounds, but less often the processes through which normative behaviors operate.
- This gap matters in creative contexts because T2I systems are increasingly used for ideation and production, while users may struggle to express thoughts in words or separate entangled concepts.
- The study examines how creative practitioners new to diffusion-based T2I tools perceive normativity across tasks with different specificity and how it affects them.
2 Study Design
The researchers conducted a within-subject study with 14 visual creative practitioners who had no or minimal prior T2I experience. Participants created illustrations in Stable Diffusion from two source media and then discussed perceived normativity.
- 14 visual creative practitioners with no or minimal prior T2I experience participated in the study.
- Participants used Stable Diffusion WebUI to create illustrations from two source media in a fixed-order within-subject design.
- The first task used a 30-second hiking soundscape, while the second used a news article about a man and his grandmother visiting U.S. national parks.
- The story task represented an uncommon older grandmother-grandson pairing that was expected to be difficult for generative AI to illustrate.
- Fixed task ordering preserved exploratory freedom in the first task but introduced possible order, practice, and fatigue effects.
- Interviews addressed expressive freedom, system assistance, and perceived neutrality, and were analyzed through inductive thematic analysis with consensus coding.
3 Highlights from our Analysis
Participants perceived Stable Diffusion’s normativity through repeated patterns, stereotypical imagery, and prompt details that were omitted or added unsolicited. These behaviors contributed to disempowerment and compromises to initial creative concepts.
- Participants identified three normative patterns: stereotypical imagery, invariant responses to prompts, and omitted or unsolicited added details.
- Participants associated stereotyped imagery with Western cultural and middle-class norms and a commercial visual culture.
- Repeated similarities in layout, perspective, and framing revealed invariant compositional behavior across outputs.
- In the more specific story task, the model often omitted the male character or made him the same age as the older woman, while ignoring style cues.
- With fewer prompt details, outputs conformed more closely to normative representations, including an older woman with white hair, glasses, and a cane.
- Participants experienced disempowerment and often abandoned aspects they perceived as uncontrollable.
- They compromised initial concepts by adapting them to perceived system capabilities, using conventional prompts, or iterating on the first generated result.
4 Discussion
The discussion frames normative T2I behavior as a challenge to creative empowerment and develops prompt transparency and XAI as possible responses. It also identifies default depictions, user frustration, and the need for artist-centered research as important concerns.
- XAIxArts is presented as an opportunity to move beyond default behavior and support artist empowerment in creative contexts.
- Normative behavior favored default depictions, which can simplify visual representation and restrict visibility of alternative viewpoints.The discussion connects adherence to defaults with artists making do with limited abstractions and being unable to explore other possibilities.
- Fine-tuning methods and other XAI approaches are discussed as promising ways to increase control, expose model behavior, and help users understand diffusion systems.Examples include LoRA, DreamBooth, interaction-focused approaches, black-box explanations, and glass-box abstractions.
- Participants’ prompt deviations, including unexplained omissions and additions, motivate prompt transparency to clarify system capabilities, limitations, and output control.The discussion mentions extending input-attribution techniques to identify omitted elements and using generated captions to highlight prompt-output discrepancies.
- Participants frequently expressed frustration and powerlessness, while extensive tinkering was identified as necessary for finding effective adjustments.These experiences are discussed as part of a broader disempowerment challenge in generative-AI creative tool practice.
- The discussion calls for research directly with artists to develop accessible interventions addressing disempowerment, AI literacy, and creative empowerment.It characterizes investigating disempowerment in genAI-based creative practice as an important research challenge.
A Story Task Article
A grandmother and grandson traveled together to visit all 63 U.S. National Parks, creating shared memories and deepening their relationship along the way.
- Joy Ryan, aged 93, visited all 63 U.S. National Parks with her grandson Brad Ryan.
- Brad began traveling with Joy after learning she had never seen America’s deserts, mountains, or oceans.
- Their first trip to the Great Smoky Mountains inspired Brad to plan a larger journey connecting nearby parks in a giant loop.
- The pair experienced adventures including whale watching in the Channel Islands and seeing enormous trees in Redwood Forest.
- Long drives also gave Joy and Brad opportunities to share difficult experiences and open up about their lives.