Source-linked AI summary
ALADIN: All Layer Adaptive Instance Normalization for Fine-grained Style Similarity
Dan Ruta, Saeid Motiian, Baldo Faieta, Zhe Lin, Hailin Jin, Alex Filipkowski, Andrew Gilbert, John Collomosse
TL;DR
Fine-grained style search is challenging because style is difficult to define and label. ALADIN learns artistic-style embeddings from weak project-group supervision using BAM-FG, and reports state-of-the-art performance for coarse- and fine-grained search.
Problem
Learning embeddings that distinguish fine-grained artistic styles is difficult because defining and labeling style is challenging.
Source: §1 · p. 1Method
ALADIN uses an encoder-decoder that pools AdaIN statistics across style-encoder layers and learns style similarity from weak project-group cues.
Source: §1 · pp. 1–2, §3 · p. 2Results
ALADIN achieves state-of-the-art performance for both coarse- and fine-grained style search without explicit style categorization.
Source: §1 · p. 2Takeaways & Limitations
BAM-FG provides 2.62 million images in 310K fine-grained style groupings, and raw project groupings suffice to train a state-of-the-art fine-grained model.
Source: §1 · pp. 1–2Fine-grained supervision remains weak: images are not explicitly labeled to a style ontology, and project groupings serve as an implicit proxy.
Source: §1 · p. 2
Abstract
from arXiv · showhide
We present ALADIN (All Layer AdaIN); a novel architecture for searching images based on the similarity of their artistic style. Representation learning is critical to visual search, where distance in the learned search embedding reflects image similarity. Learning an embedding that discriminates fine-grained variations in style is hard, due to the difficulty of defining and labelling style. ALADIN takes a weakly supervised approach to learning a representation for fine-grained style similarity of digital artworks, leveraging BAM-FG, a novel large-scale dataset of user generated content groupings gathered from the web. ALADIN sets a new state of the art accuracy for style-based visual search over both coarse labelled style data (BAM) and BAM-FG; a new 2.62 million image dataset of 310,000 fine-grained style groupings also contributed by this work.
1. Introduction
Fine-grained style search is difficult because style lacks a complete, practical labeling ontology. ALADIN learns style embeddings from weak project-group cues, and BAM-FG supplies a large dataset of such groupings.
1. Introduction
- Fine-grained style similarity is hard to learn because defining a suitable style ontology and obtaining expert annotations are difficult.Prior work therefore focused on coarse-grained discrimination of a limited number of styles [17].Source: §1 · p. 1
- ALADIN uses an encoder-decoder that pools AdaIN statistics across style-encoder layers to embed both fine- and coarse-grained artistic style.It explicitly disentangles content and style, unlike prior stylization networks that performed poorly as similarity measures.Source: §1 · p. 1
- BAM-FG contributes 2.62 million artwork images organized into 310K fine-grained style groupings gathered from Behance.net.Project co-occurrences act as weak style cues; crowd annotation partitions and cleans the data into 1.62 million images and 135K groups.Source: §1 · pp. 1–2
- ALADIN achieves state-of-the-art performance on coarse- and fine-grained style search without explicit style categorization.The model is trained with supervised contrastive learning, using weak project-group cues.Source: §1 · p. 2
- The supervision is weak because project groupings provide an implicit proxy rather than explicit fine-grained style labels.Raw groupings suffice to train a state-of-the-art model, while cleaned groups support evaluation and further discrimination.Source: §1 · p. 2
2. Related Work
Prior visual-style research has emphasized synthesis and classification, while learned search representations have generally relied on coarse labels. This work instead uses an encoder-decoder architecture for style search with weak labels and fine-grained discrimination.
2. Related Work
- Style research has focused mainly on synthesis and classification, including style transfer methods that use Gram-matrix or AdaIN feature statistics.AdaIN was built on instance normalization and uses mean-variance statistics between content and style features [14] [10].Source: §2 · p. 2
- Encoder-decoder style-transfer models encode style through architectural choices, whereas this work applies encoder-decoder models to learned search representations.MUNIT disentangles content and style via AdaIN [14] [15], and a similar architecture was later used to swap styles between images [26].Source: §2 · p. 2
- Existing learned style-search representations use explicit coarse-grained labels, while this work targets fine-grained style representation learning with weak labels.Prior approaches used triplet-based deep metric learning or classification labels [17].Source: §2 · p. 2
3. Learning Fine-Grained Style Similarity
ALADIN learns fine-grained style embeddings from weak project-group co-membership, combining multi-layer AdaIN features with reconstruction and contrastive training. BAM-FG provides the project-group data, including raw and crowd-curated partitions.
3. Learning Fine-Grained Style Similarity
- The learning goal is a fine-grained style representation trained from group co-membership rather than direct labels from a style ontology.The section frames this as a weak proxy for style similarity.Source: §3 · p. 2, §3.2 · p. 3
3.1. ALADIN Architecture
- ALADIN aggregates AdaIN statistics from multiple style-encoder layers into a search embedding, with a mirrored decoder and a projection network for both model variants.The projection network produces a 128-D L2-normalized output.Source: §3.1 · pp. 2–3
3.2. Training with Implicit Project Groups
- Project-group co-membership supplies weak supervision for contrastive learning, while reconstruction loss is added to train the encoder-decoder.The reconstruction loss uses L1 distance and is weighted by λ = 10−2.Source: §3.2 · pp. 3–4
3.2.1 Logit Accumulation
- Logit accumulation makes a batch size of 1024 fit on a single GPU with 12GB VRAM by computing logits in chunks and backpropagating through them sequentially.The method retains gradients at the logits level, then re-forwards chunks to compute model gradients.Source: §3.2.1 · p. 4
3.3. BAM-FG: Behance Artistic Media Fine-Grained dataset
- BAM-FG contains 2.62 million digital artworks in 310K project groups; its raw partition comprises 1M images in 175K projects used for training.The dataset uses Behance project co-occurrence as a weak cue for style similarity.Source: §3.3 · p. 4
- BAM-FG-CN crowd-curates 1.62M images in 135K groups into style-consistent groupings, with consensus levels CN = 1–5 used to derive evaluation groupings.Higher consensus yields greater data quality but lower data volume.Source: §3.3 · p. 4, Figure 3 · p. 4
3.3.1 Cleaning Style Groups (BAM-FG-CN)
- For BAM-FG-CN, 1,073 workers each received tasks sent to five workers; graph-based voting thresholds pairwise co-selection to form style-coherent subgroups.At CN = 3, one raw project group is partitioned into three subgroups.Source: §3.3.1 · p. 4, Figure 3 · p. 4
4. Experiments and Discussion
Experiments compare coarse- and fine-grained retrieval across BAM/BAM-X and BAM-FG, testing alternative losses, data conditions, fused embeddings, and user judgments. ALADIN-L leads the reported fine-grained benchmark comparison, while later experiments examine fusion and retrieval quality in user studies.
4.1. Datasets and Partitions
- BAM-FG-C3 supplies 78K fine-grained test images; BAM-FG-CN contains 1.62M crowd-cleaned images, while BAM-X provides 140K images across 14 coarse-style classes.The clean data is split into 115K training groups and 20K test groups; C3 is the majority-consensus level with the highest data volume.Source: §4.1 · p. 5
4.2. Baseline Methods and Losses
- Comparisons cover pretrained style features, ResNet50, DML, and MUNIT, alongside triplet and listwise losses, self-supervision, and project-membership classification.The baselines include MUNIT style codes trained unsupervised or with the contrastive scheme.Source: §4.2 · pp. 5–6
4.3. Evaluating Coarse and Fine grained Retrieval
- BAM-FG IR-1: 56.89 establishes ALADIN-Contrastive as state of the art in fine-grained retrieval; baselines perform poorly on fine-grained discrimination and generalization to new styles.Coarse-grain performance on BAM and BAM-X is satisfactory for baselines.Source: Table 1 · p. 5, §4.3 · p. 6
4.6. Visualizing Disentanglement
- 60.76% on noisy data and 62.18% at cleaned C5 are the best fused fine-grained search results.ALADIN correlates more strongly with lower-level features such as color, while the discriminative model shows greater correlation between content and style.Source: Table 4 · p. 7, §4.6 · p. 7
4.3. Evaluating Coarse and Fine grained Retrieval
- BAM-FG IR-1: 26.70 for reconstruction-only ALADIN-L on BAM-FG-Raw exceeds prior baselines, even without supervision.The unsupervised model is trained using reconstruction loss only.Source: §4.3 · pp. 6–7
4.4. Evaluating weak supervision
- ALADIN’s dual reconstruction/contrastive loss outperforms alternative weak-supervision losses on noisy project groupings and generalizes to coarse styles without coarse training labels.The comparisons include triplet, listwise, and contrastive losses on BAM-FG-Raw.Source: §4.4 · p. 7
4.5. Ablation Studies
- Up to 5% gains at C5 persist despite lower training volume; projection heads help both models, listwise beats triplet, and hard-negative mining is ineffective.The data-volume study also evaluates reductions of 25–75% of BAM-FG-Raw data.Source: §4.5 · p. 7
4.7. User Evaluation of Style Similarity
- In 1,000-query, five-worker studies, fused embeddings achieve the highest accuracy, including when same-style-group results are held out.For multi-image queries, mean-pooled encoder features represent the query group and are suitable for querying.Source: §4.7 · pp. 7–8, Table 5 · p. 8
5. Conclusion
ALADIN learns fine-grained artistic-style similarity from weak project-grouping cues and sets state-of-the-art performance in fine- and coarse-grained style discrimination. The work also contributes BAM-FG, a dataset of 2.62 million images in 310,000 groupings.
5. Conclusion
- ALADIN sets state-of-the-art performance for fine-grained discrimination, improving further with cleaned grouping data and fusion, and for coarse-grained discrimination without explicit style labels.The conclusion reports the coarse-grained result alongside the fine-grained result.Source: §5 · p. 8
- The encoder-decoder architecture learns fine-grained artistic-style similarity using weak supervision and contrastive training made practical by logit accumulation.Source: §5 · p. 8
- BAM-FG comprises 2.62 million images sampled from 310,000 user-generated groupings, whose co-occurrences serve as an implicit proxy for fine-grained style similarity.Source: §5 · p. 8