Source-linked AI summary
Measuring Similarity between Artistic and AI Generated Images using Siamese Neural Networks
Diego Castro Elvira, Navil Pineda Rugerio, Jesús García-Ramírez, Cecilia Reyes-Peña, Ricardo Ramos-Aguilar
TL;DR
The paper addresses how to measure similarity between original artworks and AI-generated counterparts, a question tied to originality and possible stylistic imitation. It builds paired artistic datasets and trains a Siamese model with frozen CLIP encoders, refined embeddings, cosine similarity, and triplet-based contrastive learning. The approach achieves high triplet discrimination, including 99.34% triplet accuracy and clear separation of positive and negative similarity distributions.
Problem
The study addresses limited specialized models for measuring similarity between original artworks and AI-generated counterparts across diverse artistic genres, motivated by originality concerns.
Method
The paper builds paired original/generated datasets and trains a Siamese network with frozen CLIP encoders, refined embeddings, cosine similarity, and triplet-based contrastive learning.
Results
99.34% triplet accuracy was achieved, with clear separation between positive and negative similarity distributions.
Takeaways & Limitations
The learned embeddings can distinguish artistic styles and may support assessment of AI-generated image fidelity and authenticity.
Takeaways & Limitations
Similarity remains moderate for some transformations, including rotation and recoloring, indicating partial robustness to visual distortions.
Abstract
from arXiv · showhide
AI-generated art has sparked debates around potential plagiarism, as these images may closely resemble existing artworks. This research quantifies the similarity between original pieces and AI-generated counterparts, particularly those produced by the Stable Diffusion XL Refiner 1.0. We use Siamese Networks with frozen CLIP encoders and cosine similarity optimized through triplet loss. A dataset of paired original and generated images was built using image-to-image generation and custom prompts, enriched with semantic descriptors and BLIP-2 captions. Prior studies report up to 81\% style replication and 90\% visual similarity. Our results show high discriminative performance: training accuracy reached 99.9\%, and the best model configuration achieved 99.4\% test accuracy with strong inter-class separation ($δμ$ = 0.677), demonstrating the effectiveness of our semantic-visual embeddings.
1 Introduction
The study addresses similarity and originality concerns for AI-generated art by comparing original and generated images with Siamese networks. It constructs diverse paired datasets and reports high triplet discrimination using frozen CLIP-based embeddings.
- AI-generated artworks can resemble human-made art, motivating similarity measurement to support digital-art authentication.
- 81,444 AI-generated images were derived from WikiArt paintings using BLIP2- and LLaMA 3 8B-created prompts with Stable Diffusion XL Refiner 1.0.
- The paired datasets include generated images, original images enriched with genre labels and BLIP2 descriptions, and uniformly resized versions at 1024×1024 and 768×768 pixels.
- The best model configuration achieved 99.4% triplet accuracy and separated matched from mismatched similarity scores.
- The study combines image-to-image generation with Siamese convolutional networks to assess semantic similarity between original and AI-generated artworks.
- The dataset spans multiple artistic genres and was developed for AI-generated-art analysis, image similarity, and style-transfer research.
2 Measuring the similarity between Artistic and AI-Generated images
The method prepares paired artistic data, generates positive counterparts and negative examples, and compares triplet embeddings through cosine similarity. Frozen CLIP representations are refined with trainable layers and optimized so positives outrank negatives.
- The pipeline prepares WikiArt images by cleaning metadata and adapting images to Stable Diffusion XL Refiner 1.0 input resolutions.
- Stable Diffusion XL Refiner 1.0 generates counterparts from BLIP2-based customized prompts using moderate or radical transformations.
- Each triplet contains an original anchor, its Stable Diffusion-generated positive counterpart, and a negative image from a different class, style, or artistic context.
- The Siamese architecture processes each image with a frozen pretrained CLIP encoder that produces a 512-dimensional visual-semantic vector.
- Trainable normalization, transformer, projection, and dropout layers refine CLIP vectors before cosine similarities are computed for positive and negative pairs.
- InfoNCE-based contrastive loss encourages higher anchor-positive similarity and lower anchor-negative similarity.
- Triplet accuracy counts cases where s+ exceeds s−, while δs = E[s+] − E[s−] measures separation between similar and dissimilar pairs.
3 Experimental Results
The study constructs paired WikiArt and Stable Diffusion XL Refiner 1.0 datasets and evaluates Siamese embeddings for semantic and visual similarity. Training metrics, configuration comparisons, similarity distributions, and inference cases show strong discrimination alongside partial robustness limits under visual transformations.
- Training results: 99.9% train triplet accuracy was reached after rapid early improvement, while validation accuracy rose from approximately 98.7% to approximately 99.5% before a slight final decline.
- Model comparison: The 70-15-15 split achieved the best results across triplet accuracy and average inter-class separation, indicating more discriminative learned representations.
- Similarity analysis: Positive similarities clustered near 1 and negative similarities nearer 0, with minimal curve overlap indicating clear separation between similarity classes.
- Inference cases: The original–generated counterpart case produced similarity close to 1, unrelated images scored 0.2841, and rotated-recolored similar images scored 0.7489.The rotated-recolored case demonstrates partial robustness to visual distortions, while the examples also expose limitations involving rotation and color manipulation.
4 Conclusions and Future Work
The study measures semantic similarity between original artworks and AI-generated counterparts using a Siamese network with CLIP-based embeddings, BLIP-2 descriptors, and synthetic triplets. It achieves high triplet accuracy and meaningful separation of similarity distributions, while slight late-training overfitting remains a limitation.
- The method combines contrastive triplet loss, a CLIP-based backbone with learnable projection layers, BLIP-2 descriptors, and Stable Diffusion XL Refiner 1.0 triplets.
- 99.34% triplet accuracy was achieved, with clear separation between positive and negative similarity distributions.
- Similarity learning without explicit class supervision effectively distinguished artistic styles.
- Slight overfitting near the end of training suggests that early stopping could improve generalization.
- The learned similarity models may help assess the fidelity and authenticity of AI-generated images and detect unauthorized stylistic imitation.