The Shape of Art History in the Eyes of the Machine

1 Introduction

• Art historians identify styles through artworks’ physical evidence together with cultural and historical context, but there is no agreed theory of how style arises or changes.Connoisseurship has been an effective means of distinguishing artists’ styles and broader style categories.Source: §1 · p. 2
• Machine classification implies learned visual features, but those features are typically not interpretable to humans, limiting what can be learned from classification results.The study uses Wölfflin’s formal, discriminative approach because its compare-and-contrast logic is suited to machine learning.Source: §1 · pp. 2–3
• Wölfflin’s formal analysis compares visual principles, separating form from subject matter and expression and proposing patterns in how styles change.His five paired principles include linear/painterly and planar/recessional.Source: §1 · pp. 2–3

Methodology

• The study analyzes CNN representations of 77K paintings using dimensionality reduction, visualization, and correlations with time and Wölfflin’s concepts.It adapts AlexNet, VGGNet, and ResNet variants to classify 20 style classes.Source: Methodology · p. 3

Main Results

• Without time information at training, learned representations arrange styles in a smooth radial chronology from Renaissance through modern styles, with later styles returning near Renaissance.The angular coordinate correlates with time at 0.69 PCC.Source: Main Results · p. 3, Figure 1 · p. 4
• The first two variation modes align with Wölfflin’s plane–recession and linear–painterly concepts, while the angular coordinate correlates with time.In Figure 1, the horizontal axis correlates with plane versus recession at -0.50 PCC, and the vertical axis with linear versus painterly at 0.36 PCC.Source: Main Results · p. 3, Figure 1 · p. 4

2.1 Challenges with Art Style Classification

• Styles transition smoothly, can overlap within a painting, and are often retrospective labels rather than mutually exclusive classes.The study therefore confronts a mismatch between art style and discrete supervised-learning categories.Source: §2.1 · p. 5
• Resizing paintings to fixed CNN inputs can distort composition and lose brushstroke texture, and this study does not address those problems.Both composition and brushstrokes are described as essential to identifying style.Source: §2.1 · p. 5

2.2 Datasets

• The training, validation, and test sets total 76,921 WikiArt paintings; the study reduces the original 27 style classes to 20 and excludes the noisy symbolism class.The labels are described as noisy, but the collection is the largest publicly available one; the split is 85%, 9.5%, and 5.5%.Source: §2.2 · pp. 5–6, Table 1 · p. 6
• Visualization uses complementary Artchive and WikiArt sets, but Artchive has gaps across periods while WikiArt omits some art-historically important works and overrepresents twentieth-century and postwar art.Artchive includes 1,485 paintings and Wölfflin-pair ratings for 1,000; the WikiArt visualization set includes 62K dated paintings.Source: §2.2 · pp. 6–7

2.3 Studied Deep Learning Models:

• The study compares AlexNet, VGGNet, and ResNet architectures and variants, adapting them for style classification.The networks were originally developed for ImageNet object recognition.Source: §2.3 · p. 7

2.4 Increasing the interpretability of the representation

• After training, the authors add fully connected layers with 1,024 and 512 nodes and fine-tune the models to constrain representation dimensionality without reducing classification accuracy.Gradually reducing layer size produced a smoother, more interpretable representation; adding the layers before training typically did not converge to similar accuracy.Source: §2.4 · pp. 7–8
• The authors examine activation variation with PCA and ICA, nonlinear manifolds with LLE, and correlations between activation dimensions, time, and Wölfflin concepts.These methods provide different views of learned representations through linear and nonlinear dimensionality reduction.Source: §2.4 · p. 8

3 Quantitative Comparative Results

• Pretrained and fine-tuned networks achieved 7% to 18% higher accuracy than their counterparts trained from scratch.Scratch-trained art-domain filters differed from ImageNet filters, with no oriented-edge-like filters observed except a horizontal edge filter.Source: §3 · p. 8, Table 2 · p. 8, Figure 2 · p. 9
• Increasing depth from AlexNet’s 5 convolutional layers to ResNet’s 152 added no more than 5% accuracy; VGGNet’s 13 layers improved 2–3% over AlexNet.Increasing VGGNet depth did not improve results, and a 50-layer ResNet trained from scratch performed worse than 5-layer AlexNet.Source: §3 · p. 9
• Experiments showed that adding extra fully connected layers while gradually reducing their node counts forced networks to achieve smoother, more interpretable representations; in most cases, these added layers enhanced accuracy.Source: Effect of adding layers with reduced… · p. 9
• Added reduced-dimension layers reduced AlexNet’s 95%-variance subspace to 9 or 10 dimensions from 201 or 397, while retaining around 60% in the first two.VGGNet’s subspace fell to 7 dimensions for 95% variance, and its classification accuracy improved by 2–4%.Source: Effect of adding layers with reduced… · pp. 9–10, Table 3 · p. 10
• ResNet’s 95%-variance subspace shrank from 491 to 6 dimensions, with 74% of variance retained in the first two dimensions.The added reduced-dimension layers preserved classification accuracy.Source: Effect of adding layers with reduced… · p. 10, Table 3 · p. 10, Figure 3 · p. 10

Few Factors Explain the Characteristics of Styles

• Fewer than 10 variation modes explained over 95% of variance across studied models; in WikiArt, 10, 9, and 7 dimensions retained 95% for AlexNet+2, VGGNet+2, and ResNet+2.The first two modes explained 60%–74% of variance in Visualization Set I; visualizations placed pre-1900 art on a two-dimensional plane.Source: Few Factors Explain the… · p. 11

Smooth Temporal Evolution of Style

• Paintings formed a smooth temporal progression despite training only on discrete style labels, without creation dates, historical context, or relations between styles.For AlexNet+2, the angular coordinates correlated with time at 0.69 PCC; ResNet’s radial coordinates had the strongest temporal correlation, 0.76 PCC.Source: Smooth Temporal Evolution of Style · pp. 11, 13, Figure 4 · p. 12

Relation to Wölfflin’s Pairs

• Across all tested networks, the first variation mode correlated most with planar versus recession, while the second correlated most with linear versus painterly, consistently across learned representations.The cited results report -0.50 Pearson correlation for planar versus recession and 0.36 for linear versus painterly; projecting into these two modes explains the representation’s radial temporal correlation.Source: Interpretation of the Modes of… · p. 14
• AlexNet+2’s fourth and fifth dimensions separated Renaissance from Baroque, with the fifth dimension correlating with Wölfflin’s relative-versus-absolute clarity, unity-versus-multiplicity, open-versus-closed, and painterly-versus-linear contrasts.The reported correlations were 0.36, 0.30, 0.28, and 0.16 PCC, respectively.Source: Interpretation of the Modes of… · p. 14, Figure 5 · p. 15

Discovering Limitations of Wölfflin’s Concepts

• Some learned modes did not correlate with Wölfflin’s concepts; the fifth embedded-manifold dimension separated Impressionism from Post-Impressionism with almost zero linear correlation to those pairs.The authors therefore state that this separation is not interpretable in terms of Wölfflin’s pairs.Source: Discovering Limitations of W¨olfflin’s… · p. 17

Identification of Representative Artists

• Representations repeatedly placed distinctive artists at the extremes of style-aligned dimensions, quantitatively highlighting artists such as Van Eyck, Dürer, Rubens, Rembrandt, Van Gogh, and Cézanne.In the VGGNet visualization, the first three modes retained 77% of variance; the reported artist patterns were also consistent in the 62K-painting WikiArt visualization.Source: Identification of Representative Artists · pp. 17–18
• ICA factorized the VGGNet activation subspace retaining 95% of variance into seven components, some aligned with styles and others contrasting styles.These axes also made representative artists quantifiable as extreme points; Fast ICA’s over-completeness limitation meant only some axes aligned with styles.Source: Identification of Representative Artists · p. 17

Interpretations of the Activation Manifold and Understanding Influences

• The activation manifold connected Cézanne’s Post-Impressionist works toward Cubism and abstraction, while some Renaissance works by El Greco and Dürer connected to modern art.Both connections appeared across representations, though in different forms; Cubism often appeared as a singularity.Source: Interpretations of the Activation… · pp. 19, 21

• Between 7 and 10 factors explain 95% of variance across the studied datasets and trained models, indicating that few factors capture style variation.Source: §5 · pp. 21, 23
• The learned representation shows radial temporal progress aligned with Wölfflin’s plane-versus-recession and linear-versus-painterly concepts, while visual similarity helps produce smooth chronology without temporal training cues.The loop closes near Renaissance and 20th-century styles, which share linearity and planarity in their form.Source: §5 · p. 23
• Activation-manifold analysis quantified Cezanne’s bridge between Impressionism and Cubism-Abstract art, as well as Renaissance-modern connections through El Greco, Dürer, Raphael, and others.Source: §5 · p. 23
• The machine consistently identified distinctive representative artists at the extremes of style-aligned dimensions, including Raphael, Monet, Cezanne, Picasso, and Kandinsky.Source: §5 · p. 23
• The loop between Renaissance and modern styles suggests that the networks’ discrimination reflects form-related concepts rather than subject matter.Source: §5 · p. 23
• The study presents machine learning as a way to discover patterns in art history and empirically revisit formal methods, while confirming existing observations and offering connections for further exploration.Source: §5 · pp. 21, 24