PaperlayerView exact revision on arXiv

Source-linked AI summary

The Prompt Artists

Minsuk Chang, Stefania Druga, Alex Fiannaca, Pedro Vergani, Chinmay Kulkarni, Carrie Cai, Michael Terry

arXiv:2303.12253v1cs.HC

TL;DR

As text-to-image models spread, the artistic practices of their prolific users remain an open area of inquiry. The paper surveys users and studies 11 prominent artists, finding that prompts and customizable templates can be artistic outputs, while users pursue distinctive results through language and model glitches.

  • Problem

    The paper investigates the practices, motivations, and goals of artists using recent text-to-image models.

    Source: §1 · p. 2

  • Method

    The authors survey internal text-to-image users and interview and observe 11 prolific artists.

    Source: §1 · p. 2, §3 · p. 5

  • Results

    Artists treat prompts and customizable prompt templates as art, seek distinctive outputs through specialized vocabulary, and sometimes develop styles from model glitches.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    The findings suggest design directions for discovering novel language and capabilities, validating originality, and supporting prompt templates as interactive artifacts.

    Source: §8 · p. 17

    The participant pool came from one US-based corporation and did not cover how other communities and collaborations shape text-to-image use in depth.

    Source: §7.5 · pp. 16–17

Abstract

from arXiv · show

This paper examines the art practices, artwork, and motivations of prolific users of the latest generation of text-to-image models. Through interviews, observations, and a user survey, we present a sampling of the artistic styles and describe the developed community of practice around generative AI. We find that: 1) the text prompt and the resulting image can be considered collectively as an art piece prompts as art and 2) prompt templates (prompts with ``slots'' for others to fill in with their own words) are developed to create generative art styles. We discover that the value placed by this community on unique outputs leads to artists seeking specialized vocabulary to produce distinctive art pieces (e.g., by reading architectural blogs to find phrases to describe images). We also find that some artists use "glitches" in the model that can be turned into artistic styles of their own right. From these findings, we outline specific implications for design regarding future prompting and image editing options.

1 INTRODUCTION

As text-to-image models became more accessible, creators formed communities around sharing prompts, designs, and images. A survey and interviews with prolific users reveal artistic practices in which prompts and templates are themselves valued outputs, alongside efforts to develop distinctive styles and verify originality.

1 INTRODUCTION

  • The study combines a survey of text-to-image users with interviews and observations of 11 prolific artists at a US-based technology company.The inquiry is scoped to interfaces that accept text prompts as input.Source: §1 · p. 2
  • Artists treat a concise, descriptive prompt as part of an artwork and a prompt template with fillable slots as an artwork in its own right.A template can encapsulate an artistic vision while remaining customizable by later users.Source: §1 · pp. 2–3
  • Users seek distinctive outputs through specialized vocabulary, including terms found in domain-specific blogs, and some turn model glitches into styles.One artist explored imperfect mirror reflections across multiple pieces.Source: §1 · p. 3
  • Interviewed artists place a premium on originality, and some use image search to check whether generated outputs are unique.Source: §1 · p. 3
  • The findings suggest tools for discovering novel language and model capabilities, validating originality, and making prompt templates standalone interactive artifacts.The authors say these implications apply beyond the internal models studied.Source: §1 · p. 3

2 RELATED WORK

Prior research spans generative art, creativity-support tools, human-AI co-creation, and debates over AI's role as an artistic medium. This paper examines how artists use recent text-to-image models within that landscape, while distinguishing diffusion and autoregressive approaches.

2 RELATED WORK

  • Earlier generative methods enabled novel artistic styles, but provided end users little control over outputs beyond the training-data scope.Source: §2 · pp. 3–4

2.1 AI in Creativity Support Tools & Human-AI Co-Creation

  • Creativity-support research frames text-to-image systems as generators, while co-creation work describes interaction patterns that include exploration and curation.The paper examines practices and goals developed alongside recent text-to-image models.Source: §2.1 · pp. 4–5

2.2 Generative AI as an Artistic Medium

  • Debates variously frame generative models as artists, tools, or agents, while this study finds dedicated users employ them as tools to explore themes and concepts.Some users develop processes such as locating domain-specific terminology to pursue their goals.Source: §2.2 · p. 5

2.3 Diffusion and Auto-Regressive Models

  • Diffusion models generate images by reversing a noising process, whereas autoregressive models treat text-to-image generation as translation from text tokens to image tokens.Study participants used both types of models.Source: §2.3 · p. 5

3 STUDY DESIGN

The study pairs an internal-user survey with interviews and observations of prolific text-to-image users. Its qualitative analysis combines transcript review, observations of participants' nonverbal interactions, and group interpretation of themes.

3 STUDY DESIGN

  • The study surveyed internal users about model use and interviewed and observed 11 power users in 50-minute sessions.The models were anonymized and described as having capabilities comparable to state-of-the-art systems.Source: §3 · pp. 5–6

3.1 Participants

  • Interviewees were recruited from survey respondents who reported repeated artwork creation and substantial recent model use, alongside three prominent artists.Eight survey respondents and three prominent community artists were recruited; participants were active in external communities as well.Source: §3.1 · p. 6

3.2 Interview structure

  • Interviews asked participants to create an image, discuss successful and unsuccessful work, revise another person's prompt, and envision future model uses.Sessions were remote, screen-recorded, and limited to text-prompt interfaces.Source: §3.2 · p. 6

3.3 Qualitative Data Analysis

  • The first two authors independently reviewed 164 pages of transcripts, then all authors held two interpretation rounds to agree on themes.Analysis also considered nonverbal interactions, prompts, and participant-created images.Source: §3.3 · p. 6

4 SURVEY RESULTS

Survey responses describe varied motivations and sustained use of text-to-image models, alongside requests for greater prompting guidance and control. Respondents also reported strategies for shaping outputs through style, camera, and domain-specific language.

4 SURVEY RESULTS

  • Among 160 respondents, 79 (49%) used the models to create art, 33 (21%) for creative work, and 126 (79%) out of curiosity.Source: §4 · p. 6
  • 20% of 157 respondents used a model for one or more hours per session, while 53% reported 10 minutes to an hour.Another 27% reported less than 10 minutes per session.Source: §4 · p. 7
  • 76% of respondents requested a prompt-building guide, 75% wanted to fork and remix images, and 63% desired greater control over models.Respondents also requested features such as bookmarking and sharing outputs.Source: §4 · p. 7
  • Reported prompting strategies included naming artistic styles or eras, camera and aperture terms, and domain-specific vocabulary.Source: §4 · p. 7

5 PROMPT ARTISTS: STYLES, MOTIVATIONS, PRACTICES

The section profiles three highly active TTI creators and summarizes motivations, styles, and practices across 11 interview participants. Their examples show how artists develop distinctive visual concepts and treat prompting as a creative practice.

5.1 Shai Noy: The Explorer (A1)

  • Shai Noy developed super-macro photography and fashion images made from unusual materials such as wood, grass, brick, or ice.Noy had produced thousands of images despite having no visual-arts or design training.Source: §5.1 · p. 7
  • For Noy, discovery and sharing discoveries with others were rewarding parts of making art with TTI models.Noy valued exploring concepts first and sharing findings with the community.Source: §5.1 · p. 7

5.2 Irina Blok: The Art Director (A2)

  • Irina Blok pursued impossible objects and mash-ups, seeking images that differed from the models’ training examples.She described creative prompts as a way to defy the rules of the training set.Source: §5.2 · p. 8
  • Blok also crafted prompt templates with fill-in slots, designed to reliably produce pleasing results for others.One template describes a fantasy house shaped like a user-selected object and located in a user-selected place.Source: §5.2 · p. 8
  • Blok viewed the model as both an artist she directed and a tool whose language she had to learn, emphasizing conceptual prompt-making over visual execution.She described the process as requiring a thought exercise and acknowledged a lack of full control.Source: §5.2 · p. 8

5.3 Dan Smith: The Social Commentator (A3)

  • Dan Smith sought climate-crisis imagery that could convey a message and evoke feeling, including nature in unfamiliar situations and hybrid animals.Smith also valued images that would be difficult to create or visualize even with advanced Photoshop skills.Source: §5.3 · pp. 8–9
  • Smith selected outputs for near expert-level composition, photorealism, or artistry, and used “tall grass” to conceal awkward animal limbs and paws.The strategy reflects an understanding of model limitations and a way to cope with them.Source: §5.3 · p. 9

5.4 Summarizing Motivations, Styles, and Practices

  • Across participants, TTI use was fun and supported creative exploration of personal interests, while spotlighted artists pursued themes in greater depth than other participants.Examples of personal-interest uses included Swiss trains, new forms of currency, and presentation clip art.Source: §5.4 · pp. 9–10

6 THEMES: ART BEYOND IMAGES, DISCOVERING UNIQUE POINTS, AND VALIDATING ORIGINALITY

Artists treat prompts and prompt templates as artistic outputs alongside images, while pursuing distinctive model capabilities and checking whether successful outputs are original.

6.1 Prompt as Art

  • Some artists treat the prompt and resulting image as a mutually reinforcing artwork, making the prompt both source material and a possible title.The pairing can also convey the idea of the image and how it was made; artists vary between descriptive and simple prompts, and even value unexpected outputs from random strings.Source: §6.1 · p. 11

6.2 Prompt Templates As Art

  • Prompt templates encode an artistic concept in fillable slots, aiming to let others customize outputs while preserving a recognizable composition and reliable quality.Users’ slot choices can interact with the rest of the prompt: strawberry houses in different locations acquire different styles, enabling varied explorations.Source: §6.2 · pp. 12–13

6.3 Discovering Unique Points in the Model’s Latent Landscape

  • Artists seek unfamiliar model capabilities by researching distinctive vocabulary, iteratively steering prompts, and sometimes turning model glitches into styles.Examples include using architectural terms or unusual words, and developing imperfect reflections or cat-back faces into artistic motifs.Source: §6.3 · pp. 13–14

6.4 Validating Originality

  • Some artists use image search on compelling outputs to check artifact originality, partly because they are concerned models may memorize training data.The check concerns similarity of the artifact, not direct validation of the originality of the artistic concept.Source: §6.4 · p. 14

7 DISCUSSION

The discussion derives text-only design ideas from artists’ goals of novelty, originality checks, and reusable templates. It also distinguishes prompt artists from practitioners seeking finer controls, while noting limits in the study’s scope.

7 DISCUSSION

  • The study documents artists’ goals beyond image production, including prompts and prompt templates as artifacts in their own right.The discussion frames its design implications around novelty, originality validation, and reusable templates, and limits the discussion to text input without image post-processing.Source: §7 · p. 15

7.1 Aiding Novelty

  • Novelty tools could surface uncommon terms through thesauri, search, language models, or training-data associations, and visualize how common prompt tokens are.For template slots, interfaces could also show common fill-in terms; unusually novel inputs may fall outside the training distribution and produce unpredictable results.Source: §7.1 · p. 15

7.2 Validating Originality

  • Originality tools could streamline external image search and search training data for similar generated images or prompt content.These proposals address both the generated artifact and, potentially, the input prompt.Source: §7.2 · p. 15

7.3 Materializing Prompt Templates

  • Prompt templates could become first-class interactive objects, letting users explore combinations of slot values or interpolate between text embeddings.The proposed interpolation occurs in text space, so it connects linguistic concepts rather than necessarily morphing between image-space shapes.Source: §7.3 · p. 16

7.4 Prompts and TTI Models as an Art Medium

  • Prompt artists may embrace text-only input as a defining feature, while creative professionals may want finer controls such as direct image editing.The authors suggest that, if non-design experts use models to visualize artifacts, feedback and collaboration features could become more important.Source: §7.4 · p. 16

7.5 Limitations

  • The study draws participants from one large US-based corporation and does not document in depth how other communities or models shape practice.The analysis is episodic rather than longitudinal, and participants’ observed interactions do not definitively establish their conceptions of model behavior.Source: §7.5 · pp. 16–17

8 CONCLUSION

The conclusion identifies a vibrant practice around recent text-to-image models and calls for tools supporting novelty, originality validation, and interactive prompt templates.

8 CONCLUSION

  • The authors conclude that text-to-image practice includes notable outcomes beyond images, and that tools should support novel outputs, novelty validation, and interactive prompt templates.They also distinguish prompt artists who embrace prompt-only creation from practitioners who may want finer input and editing controls.Source: §8 · p. 17

Related papers

Loading 2303.12253v1…