Source-linked AI summary
Understanding and Creating Art with AI: Review and Outlook
Eva Cetinic, James She
TL;DR
Research on AI and art lacks an integrated account of how AI supports both art understanding and art creation. This paper reviews datasets, computational analysis tasks, generative-art methods, and practical and theoretical issues, finding expanding applications alongside unresolved questions about context, autonomy, authorship, and perception.
Problem
AI’s role in understanding and creating art raises unresolved questions about computational interpretation, creativity, autonomy, authorship, and art-historical context.
Method
The paper provides an integrated review of AI for analyzing digitized artwork collections and generating novel artworks.
Results
The review covers artwork datasets and analysis tasks alongside generative-art systems and debates concerning AI Art’s practical and theoretical dimensions.
Takeaways & Limitations
AI technologies are becoming more relevant to art analysis and production while encouraging new digital and interdisciplinary research perspectives.
Takeaways & Limitations
Current approaches to aesthetics and perception are limited because they consider visual image features without fully incorporating art-historical context.
Abstract
from arXiv · showhide
Technologies related to artificial intelligence (AI) have a strong impact on the changes of research and creative practices in visual arts. The growing number of research initiatives and creative applications that emerge in the intersection of AI and art, motivates us to examine and discuss the creative and explorative potentials of AI technologies in the context of art. This paper provides an integrated review of two facets of AI and art: 1) AI is used for art analysis and employed on digitized artwork collections; 2) AI is used for creative purposes and generating novel artworks. In the context of AI-related research for art understanding, we present a comprehensive overview of artwork datasets and recent works that address a variety of tasks such as classification, object detection, similarity retrieval, multimodal representations, computational aesthetics, etc. In relation to the role of AI in creating art, we address various practical and theoretical aspects of AI Art and consolidate related works that deal with those topics in detail. Finally, we provide a concise outlook on the future progression and potential impact of AI technologies on our understanding and creation of art.
1 Introduction
AI-and-art research spans analyzing existing art and creating new art, with visual-art applications expanding alongside digitized collections and generative methods.
- AI-and-art activities divide into analyzing existing artworks and creating new ones.
- Generative Adversarial Networks significantly accelerated AI-based visual-art production and creative exploration.
- Digitized art collections enable AI-based analysis of art history and large-scale artwork image data.
- Convolutional Neural Networks enabled automated classification, categorization, and visualization of large artwork collections.
2 Understanding Art with AI
AI-based art understanding combines digitized collections, datasets, deep-learning methods, multimodal analysis, and quantitative approaches to visual and art-historical concepts. The review also emphasizes interdisciplinary collaboration and the difficulty of modeling perception beyond visual features.
- Large-scale digitized art collections support interdisciplinary research and computational exploration of artworks across museums and galleries.
- 2.1 Art Collections as Data Sources: Well-annotated datasets are necessary for applying deep-learning models to artwork classification, retrieval, object detection, and other tasks.
- 2.2 Artwork Classification: CNN-based methods advanced artwork classification by using learned visual features for artist, style, and genre recognition.
- 2.3 Object Detection and Similarity Retrieval: Deep neural networks support artwork content recognition, object retrieval, and similarity-based search across painting collections.
- 2.4 Multimodal Analysis: Multimodal research links artwork images with textual descriptions for retrieval and visual question answering.
- 2.5 Knowledge Discovery in Art History: Computational analysis can quantify art-historical concepts such as style transformation and representativity using deep-network representations.
- 2.5 Knowledge Discovery in Art History: Interdisciplinary collaboration between computer science and art history is important for developing computational methods that extend digital art-history research.
- 2.6 Aesthetics and Perception: Computational aesthetics and emotion modeling remain challenging because artwork importance and perception depend on art-historical context beyond visual features.
3 Creating AI Art
AI art developed through advances from image stylization and GANs to text-to-image systems, while raising questions about novelty, autonomy, authorship, ethics, and human participation. The review connects these technologies to debates about how AI-generated works should be understood and evaluated.
- 3.1 Technological Milestones: Neural Style Transfer separates and recombines image content and style to create stylized images using CNNs.Content refers to recognizable depicted objects and figures, while style concerns visual deviation from photorealistic depiction.
- 3.1 Technological Milestones: NST outputs commonly combine existing inputs rather than constitute original artistic creations.Pairing content and style images can also be technically challenging and time-consuming when seeking meaningful results.
- 3.1 Technological Milestones: GANs generate novel visual content through competing generator and discriminator models that learn the distribution of training examples.The generator produces realistic images, while the discriminator distinguishes generated outputs from true examples.
- 3.1 Technological Milestones: AICAN modifies GAN optimization to maximize deviation from established styles while remaining within the art distribution.In exhibitions and experiments, people were often unable to distinguish AICAN-generated images from human-produced artworks.
- 3.1 Technological Milestones: AI art is increasingly produced through systems ranging from GANs to transformer-based text-to-image models such as DALL·E.DALL·E creates images from text captions for concepts expressible in natural language.
- 3.3 Novelty of AI Art: Debates about AI art address whether computers act as tools or autonomous creators, and how artists shape latent-space exploration and final works.The review also discusses authorship, copyright, ethics, interpretability, and the role of human curation in these processes.
4 Conclusion and Future Outlook
AI is expected to become increasingly relevant to both art analysis and production, while research and practice continue to face unresolved technical and methodological challenges. Advances in multimodal generative models and digital humanities are likely to further shape artistic exploration and research.
- AI is expected to become more relevant in both the analysis and production of art.
- Digital humanities programs and quantitative AI-based methods are intensifying the shift toward digital research practices in the humanities.
- Computer vision for cultural archives still faces challenges in annotation, object detection, retrieval, iconographic classification, multimodal alignment, and image understanding.
- AI is also becoming important across the curation, exhibition, and sale of traditional art, alongside digitally and AI-produced art.
- Multimodal generative models that generate images from text offer new possibilities for artistic exploration and may influence art production.