PaperlayerView exact revision on arXiv

Source-linked AI summary

Do Generative Priors Align with Human Naturalness Perception?

Taiki Fukiage

arXiv:2610.09928v1cs.CVq-bio.NC

TL;DR

Whether generative models’ native priors reflect human naturalness perception remains an open question, especially because raw losses can be confounded by image complexity. The paper probes native prediction errors with paired relational interventions and finds human-like sensitivities and item-level alignment across faces and physical scenes.

  • Problem

    It remains unclear whether generative models’ learned visual distributions capture human naturalness regularities, while raw prediction errors can be confounded by image complexity and scene statistics.

    Source: §1 · pp. 1–2

  • Method

    The study compares native prediction-loss differences for content-preserving relational interventions with human ratings across face and illumination tasks.

    Source: §1 · p. 2, F DIAGNOSTIC CONTROLS ON NATIVE… · p. 34

  • Results

    Across faces and physical scenes, loss differences reproduce human-like sensitivities and track individual judgments, peaking at r = .84 on faces and .64 on physical scenes.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Generative loss landscapes capture structured relational regularities that partially mirror human naturalness perception, while sensitivity and fine-grained alignment can dissociate.

    Source: §5 · p. 9

    Controlled relational interventions limit low-level confounds but constrain scene diversity, and extending them to unconstrained natural imagery without low-level artifacts remains challenging.

    Source: §5 · p. 10

Abstract

from arXiv · show

Visual generative models are trained to capture the probability distributions of natural images, yet whether their native priors reflect the regularities governing human perception of image naturalness remains an open question. Here, we probe these priors through native prediction errors across 25 open image and video generators. Because raw single-image losses are dominated by scene content and visual complexity, we evaluate directional loss differences using content-preserving, paired relational interventions that selectively disrupt facial configurations or physical illumination consistency while limiting changes in low-level image statistics. Across both domains, these loss differences reproduce human-like selective sensitivities and tolerances, capturing the classic Thatcher effect on faces and shape-dependent responses to illumination inconsistencies. Notably, these loss differences reliably track continuous gradations of human naturalness judgments across individual stimulus pairs (peaking at $r = .84$ on faces and $.64$ on physical scenes) and retain unique human-aligned signals even after controlling for feature distances from frozen vision encoders and standard image quality metrics. We also find that while overall sensitivity to these violations broadly covaries with human alignment across models, the two systematically decouple along denoising schedules, with alignment peaking earlier than sensitivity, revealing that human-like naturalness judgments dissociate from generic violation detection. Together, these findings demonstrate that learning visual distributions yields generative loss landscapes that capture distinct aspects of human naturalness perception.

1 INTRODUCTION

The paper tests whether generative models’ native prediction losses capture human naturalness perception, using paired relational interventions to limit low-level confounds. Across faces and physical scenes, loss differences reproduce human-like sensitivities and track individual naturalness judgments.

1 INTRODUCTION

  • Native prediction errors offer a zero-shot probe of learned image plausibility, unlike approaches that train regressors on model features or engineer attention-based metrics.For diffusion and flow models, these objectives relate to score matching and variational bounds on data log-likelihood.Source: §1 · p. 1
  • Content-preserving relational interventions target facial configurations or illumination consistency while limiting low-level image-statistic changes.Paired directional loss differences focus on relational violations while controlling for shared scene composition.Source: §1 · p. 2, Figure 1 · p. 2
  • The study evaluates 25 open image and video generators on Thatcher faces and physical-scene illumination interventions.Representative profiles show the Thatcher effect and shape-dependent shadow sensitivities, alongside specific divergences.Source: §1 · pp. 2–3, Figure 1 · p. 2
  • Across both domains, native loss differences reproduce human-like selective sensitivities and tolerances, track individual ratings, and retain unique perceptual signals beyond standard metrics and encoders.Item-level correlations peak at r = .84 on faces and .64 on physical scenes; sensitivity and alignment also decouple along denoising schedules.Source: §1 · p. 3

2 RELATED WORK

Prior work connects natural-image statistics and learned visual priors to perceptual assessment, but raw generative scores can be confounded by image complexity and composition. This paper instead probes native loss landscapes and human perceptual tolerances through controlled relational changes.

2.1 GENERATIVE MODELS AS PERCEPTUAL PRIORS

  • Feature-based generative metrics can assess perceptual quality, but do not test whether a model’s learned distribution itself aligns with human naturalness perception.The paper’s inquiry probes native loss landscapes using content-preserving relational interventions rather than intermediate readouts.Source: §2.1 · p. 3

2.2 PHYSICAL WORLD MODELS VS. PERCEPTUAL NATURALNESS

  • Objective physical validity need not match human naturalness judgments, because observers may tolerate substantial inconsistencies in lighting, shadows, and reflections.The Illumination task therefore tests correspondence with selective human tolerances and continuous naturalness ratings.Source: §2.2 · pp. 3–4

2.3 VISUAL ILLUSIONS IN DEEP NETWORKS

  • Prior illusion studies span low-level brightness, surface filling-in, face configuration, and multimodal question-answering, while this work probes biases in native generative loss landscapes.Examples include Thatcher effects in feedforward networks and hollow-face effects in structured inverse-graphics models.Source: §2.3 · p. 4

3 METHODS

The methods use controlled face and illumination stimuli, native prediction losses from image and video generators, and paired metrics designed to compare model responses with human judgments. Baselines and resampling procedures assess alignment, complementary information, and uncertainty.

3.1 STIMULUS DESIGN AND PSYCHOPHYSICAL EXPERIMENTS

  • The Thatcher and Illumination tasks use content-preserving interventions across 140 face scenes and 96 rendered scenes, with human observers rating image naturalness.Face features are reflected in upright and inverted images; illumination interventions alter shadows, reflections, or light direction while objects remain fixed.Source: §3.1 · pp. 4–5

3.2 GENERATIVE MODELS AND NATIVE LOSS PROBING

  • The study evaluates 25 open image and video models using native prediction losses averaged across model timesteps and noise draws.Paired loss changes compare modified and original images, controlling for shared scene content; raw loss magnitudes are not comparable across architectures.Source: §3.2 · p. 5

3.3 PERCEPTUAL EVALUATION METRICS

  • Task-standardized paired effects support condition profiles, while item-level Pearson correlations measure alignment and sensitivity summaries test its relationship with coarse violation detection.The analysis also uses bidirectional partial correlations to assess non-redundant signals and bootstrap resampling to estimate uncertainty.Source: §3.3 · pp. 5–6

3.4 BASELINE MODELS

  • Baseline comparisons include 12 frozen encoders and two five-metric image-quality batteries, with the best-performing encoder layer selected for each task.Bidirectional partial correlations compare human alignment for generative–encoder pairs, summarized by medians across selected models and metrics.Source: §3.4 · p. 6

4 RESULTS

Paired generative loss differences track human naturalness judgments across individual face and physical-scene stimuli, outperform evaluated baselines, and retain distinct signals after controls. Their selective sensitivities resemble human responses, but sensitivity and human alignment also diverge across conditions and denoising stages.

4.1 QUALITATIVE VALIDATION OF GENERATIVE LOSSES

  • Generative losses reproduced human-selective responses to upright Thatcher-face inversions and shape-dependent shadow rotations, but diverged on reflection color.Sensitivity to facial inversions dropped under face inversion; humans penalized reflection hue mismatches more strongly than generative models.Source: §4.1 · pp. 6–7
  • Spatial loss increases extended beyond edited regions to related scene elements, indicating sensitivity to contextual relationships.Face penalties concentrated around inverted eyes and mouths, while illumination penalties also reached nearby surfaces or unmodified objects.Source: §4.1 · p. 7, Figure 2 · p. 7

4.2 ITEM-LEVEL HUMAN ALIGNMENT

  • Median item-level correlations reached .747 on Thatcher and .334 on Illumination, peaking at .841 and .639, respectively.Condition-centered correlations remained positive for all 25 models on both tasks, while unconditioned single-image losses showed little correlation with human ratings.Source: §4.2 · p. 7, Figure 3 · p. 8
  • Top generative models exceeded encoder baselines on both tasks, while image-quality metrics showed substantially weaker correspondence with human judgments.Top-five median r was .816 vs. .660 on Thatcher and .602 vs. .492 on Illumination for generative models and encoders, respectively.Source: §4.2 · p. 7, Figure 3 · p. 8
  • Generative scores retained positive residual associations after controlling for baselines, with complementary encoder signals on Illumination.On Thatcher, median partial correlation was .676 for generative models versus .130 for encoders; on Illumination, the values were .510 and .375.Source: §4.2 · pp. 7–8, Figure 3 · p. 8

4.3 COUPLING AND DISSOCIATION OF SENSITIVITY AND ALIGNMENT

  • Across tasks, violation sensitivity correlated positively with human alignment, but associations varied by condition.Whole-task correlations were r = .663 on Thatcher and r = .821 on Illumination; light direction was near zero and surface reflections negative.Source: §4.3 · p. 8, Figure 4 · p. 9
  • Sensitivity peaked later than human alignment in 20 of 25 models on Thatcher and all 25 models on Illumination.Median within-model peak shifts were 0.141 and 0.444 on the normalized schedules, respectively.Source: §4.3 · p. 8
  • Exploratory generation-benchmark associations with human alignment were positive for Illumination image models and both tasks in video models, but weak for Thatcher image models.The authors note that these exploratory associations are based on modest sample sizes.Source: §4.3 · p. 9

5 DISCUSSION

The discussion argues that generative losses capture some human-like naturalness judgments, while their sensitivities do not fully match human perceptual weighting. The accompanying methods describe controlled stimulus construction, rating procedures, and experimental assignment across face and illumination tasks.

5 DISCUSSION

  • Generative losses capture structured relational regularities that partly mirror human naturalness perception, but sensitivity and fine-grained alignment can dissociate.The authors also note that these findings concern existing architectures and frame the diagnostic as a foundation for tracking perceptual alignment as models scale.Source: §5 · p. 9
  • Generative models capture human-like orientation dependence in face judgments and outperform layer-optimized encoders in Thatcher-task alignment.The discussion reports naturalness tracking within upright faces alone (median r = .493) and says the orientation penalty appears in ImageNet-trained JiT models without dedicated face training.Source: §5 · pp. 9–10
  • Human sensitivity to shadow and reflection inconsistencies depends on scene geometry, and generative losses track this selectivity across rotations and lighting directions.Models respond much less than humans to reflection hue shifts, suggesting that relational cues are not weighted identically to human perception.Source: §5 · p. 10
  • Larger models became more sensitive to surface-reflection violations without improving in human alignment, so sensitivity alone does not ensure human-like perceptual weighting.Whole-task alignment scaled positively with model size in exploratory comparisons, but that relationship did not uniformly extend to individual conditions.Source: §5 · p. 10
  • Controlled relational interventions limit low-level confounds but constrain scene diversity, while off-the-shelf model comparisons cannot isolate objectives from scale, architecture, or training recipes.The authors also state that observational links to generation benchmarks cannot establish whether optimizing human-aligned relational priors causally improves generation quality.Source: §5 · p. 10

A.1 THATCHER FACES.

  • The Thatcher stimuli use 140 aligned face identities, with eye and mouth features vertically inverted in place and blended to reduce seam artifacts.Both original and modified images were presented upright and after a 180° rotation.Source: §A.1 · pp. 17–18

A.2 ILLUMINATION SCENES.

  • The illumination set contains 288 paired scenes across shadow, reflection, and light-direction operations, created by rearranging existing pixels while keeping depicted objects in place.Floor regions are flipped for shadow and reflection pairs, while the central object is flipped for light direction; spatial gaps help avoid contact-edge artifacts.Source: §A.2 · p. 18
  • Shadow and reflection panels vary shape, orientation, height, and softness or floor roughness, while light-direction panels vary shape, material, height, and light angle.The appendix figures illustrate selected factor levels and pairwise comparisons rather than every factorial combination.Source: §A.2 · p. 18, Figure S3 · p. 20, Figure S4 · p. 21

B.4 PARTICIPANT DEMOGRAPHICS, SCREENING, AND RESPONSE COUNTS

The authors screened participants using agreement with other observers’ image-wise ratings, retaining 143 Thatcher participants and 105 Illumination participants. They report robustness checks showing that key aggregates and model–human correspondence were similar without screening.

B.4 PARTICIPANT DEMOGRAPHICS, SCREENING, AND RESPONSE COUNTS

  • Screening retained 143 Thatcher participants (88.3%) and 105 Illumination participants (63.6%) using task-specific cutoffs of r = .2481 and r = .3236.Cutoffs were selected at the deepest trough between peaks in estimated correlation distributions.Source: §B.4 · p. 27, Figure S9 · p. 28
  • After screening, mean independent ratings per item were 35.8 for Thatcher and 26.2 for Illumination, with illumination counts differing across intervention types.Shadow averaged 40.5 responses, reflection 26.2, and light direction 12.0.Source: §B.4 · p. 27
  • Light-direction screening excluded 69.2% of enrolled participants, whose split-half reliability and original–modified rating separation were lower than those of retained participants.The excluded group’s mean rating difference was 0.051 points, versus 0.996 points among retained participants.Source: §B.4 · p. 27
  • Without exclusions, item aggregates correlated with screened aggregates at r = .986 for Thatcher and r = .942 for Illumination.The sensitivity–alignment association was also preserved, and Light-direction model–human correlations were similar with and without screening.Source: §B.4 · pp. 27–28

C MODEL DETAILS AND EVALUATION SPECIFICATIONS

The evaluation averages native generative prediction losses across model-specific timestep grids and shared noise draws, then compares paired-image scores with fixed encoder and IQA baselines. Statistical procedures use resampling to quantify uncertainty across observers, stimuli, and model families.

C.1 GENERATIVE LOSS PROBING

  • For each model and test image, native loss averages squared prediction error over a fixed grid of timesteps and noise realizations.Image models use 100 scheduler-defined timesteps and video models 50; each uses a shared bank of 20 noise tensors across stimuli and paired images.Source: §C.1 · pp. 28–29

C MODEL DETAILS AND EVALUATION SPECIFICATIONS

  • The evaluation covers 25 generative models across pixel-space image, latent-space image, and spatiotemporal video architectures.Model-specific scheduler, identifier, and environment settings are archived with the code and configurations.Source: C MODEL DETAILS AND EVALUATION… · p. 28, Table S1 · p. 29

D NON-GENERATIVE BASELINE DETAILS AND SCORING METHODS

D.1 FROZEN ENCODER DISTANCES

  • Global encoder pair scores use unsigned cosine distance between independently extracted original and modified image representations, without fitting a human-judgment readout.Architecture-specific readout rules are held fixed across blocks and tasks, and DINOv2/v3 scores use corresponding spatial patch-token distances.Source: §D.1 · p. 30

D.2 FULL-REFERENCE IQA (FR-IQA)

  • The full-reference IQA battery evaluates distortion relative to the pristine original image.It includes PSNR, SSIM, VIF, LPIPS-VGG, and DISTS, which measure pixel error, local structure, visual information, deep-feature distance, or structural and texture similarity.Source: §D.2 · pp. 30–31

D.3 NO-REFERENCE IQA (NR-IQA)

  • No-reference IQA scores images independently, and paired score differences are oriented so positive values indicate higher quality for the original.The battery includes NIQE, BRISQUE, HyperIQA, MUSIQ, and LIQE.Source: §D.3 · p. 31

D.4 PREPROCESSING AND EXECUTION ENVIRONMENT

  • Inputs were resized to 512 × 512 pixels for IQA scoring, followed by model-specific channel normalization and default PyIQA processing.Scoring ran on NVIDIA RTX A6000 GPUs.Source: §D.4 · p. 31

E STATISTICAL PROCEDURES AND ANALYSIS DETAILS

  • Primary human-dependent estimates use a two-factor bootstrap that resamples participants and base scenes, while cross-model associations also bootstrap model families.The procedures use 10,000 draws; human split-half reliability is estimated across 10,000 participant partitions with synchronized base-scene resampling for intervals.Source: §E.1 · p. 33, §E.2 · p. 33

E.3 BIDIRECTIONAL PARTIAL CORRELATION ANALYSIS

The analysis compares generative scores with non-generative baselines using bidirectional partial correlations, then tests when native loss differences fail or succeed under low-level distortions and shadow manipulations.

E.3 BIDIRECTIONAL PARTIAL CORRELATION ANALYSIS

  • Generative and encoder candidates were selected by task-wide human-rating correlation, while all IQA metrics were retained for the baseline comparisons.The five selected models and encoder layers stayed fixed across condition breakdowns and factor profiles.Source: §E.3 · pp. 33–34
  • For each selected generative–comparator pair, the analysis computes partial correlations in both directions and summarizes each direction by the median across 25 pairs.The two directions assess residual human-aligned associations for the generative score and comparator, respectively.Source: §E.3 · pp. 33–34
  • The 95% confidence intervals repeat model and encoder selection within each of 10,000 participant-and-scene bootstrap draws.Selections are applied consistently across the overall task and condition breakdowns for each draw.Source: §E.3 · p. 34

F.1 GENERIC DISTORTIONS ON CSIQ AND ELEMENT DELETION

  • Across CSIQ distortions, loss–DMOS correlations were positive for AWGN and pink noise but reversed for blur, JPEG, JPEG 2000, and contrast reduction.The evaluation used pixel-space diffusion models to avoid confounding generative priors with VAE compression; the authors conclude native loss is not a general-purpose quality metric.Source: §F.1 · p. 34
  • Shadow deletion produced uniformly negative loss changes, whereas shadow swaps preserved low-level statistics and let more capable architectures show positive penalties for physical inconsistency.The comparison used 96 rendered scenes; deletion also reduced image complexity, which dominated the loss landscape.Source: §F.1 · pp. 34–35, Figure S10 · p. 35

F.2 SINGLE-IMAGE ABSOLUTE SCORE ANALYSIS

Standalone image losses do not consistently align with human unnaturalness ratings, whereas paired loss differences correlate positively with human paired judgments. Sensitivity and alignment covary in some stimulus conditions but not others.

F.2 SINGLE-IMAGE ABSOLUTE SCORE ANALYSIS

  • Paired loss differences correlated positively with human paired differences across all 25 models, with median correlations of .747 on Thatcher and .334 on Illumination.By contrast, raw-loss median correlations were .236 and −.090 for original and modified Thatcher images, and −.145 and −.203 for original and modified Illumination images.Source: Figure S11 · p. 36, §F.2 · p. 36

G MODEL-WISE SENSITIVITY AND HUMAN ALIGNMENT

  • The model-wise analysis distinguishes violation sensitivity from human alignment, with alignment measured as correlation between model loss differences and human paired-rating differences.The figure reports both measures across all 25 generative models and five stimulus conditions.Source: Figure S12 · p. 37, G MODEL-WISE SENSITIVITY AND HUMAN… · p. 37

H SENSITIVITY AND HUMAN ALIGNMENT ACROSS INDIVIDUAL CONDITIONS

  • Sensitivity–alignment associations were robust for inverted faces (r = .723, 95% CI [.118, .923]) and cast shadows (r = .810, [.424, .960]), but weak, uncertain, or negative elsewhere.Estimates were positive but uncertain for upright faces (r = .667, [−.215, .904]), absent for light direction (r = .051, [−.417, .570]), and negative for reflections (r = −.371, [−.704, .364]).Source: Figure S13 · p. 38, H SENSITIVITY AND HUMAN ALIGNMENT… · p. 38

I CONDITION-SPECIFIC EVALUATION AGAINST NON-GENERATIVE BASELINES

Generative scores retain positive human-aligned residual associations across conditions after controlling for non-generative baselines, while factor profiles compare model responses with human responses across scene properties.

I CONDITION-SPECIFIC EVALUATION AGAINST NON-GENERATIVE BASELINES

  • For surface reflections, both generative models and encoders retained residual associations: r(h, G | Encoder) ≈.346 and r(h, Encoder | G) ≈.584.The authors interpret this as evidence that both representations capture distinct aspects of reflection naturalness.Source: I CONDITION-SPECIFIC EVALUATION… · p. 39
  • Generative scores retained positive residual correlations with human judgments after controlling for each non-generative baseline across conditions.They accounted for substantial aligned variance beyond encoders and quality metrics for upright faces, cast shadows, and light direction; under inversion both generative and baseline residuals attenuated.Source: I CONDITION-SPECIFIC EVALUATION… · p. 38, Figure S14 · p. 39

J FACTOR-LEVEL STRUCTURE OF ILLUMINATION JUDGMENTS

  • The factor analysis uses task-pooled top-five generative models and encoders, retains all IQA metrics, and examines object shape, rotation, color, and physical parameters.The selected model sets match those used in the condition-level comparison.Source: J FACTOR-LEVEL STRUCTURE OF… · p. 40

J.1 FACTOR PROFILES AND COMPARISON METRIC

  • Factor profiles plot mean paired responses at each scene-factor level, marginalizing over other parameters, and compare model and human profiles using weighted mean squared error.Scores are scaled by each respondent’s task standard deviation without mean-centering; each of the three operations receives equal total weight.Source: §J.1 · p. 40

J.2 HUMAN PERCEPTUAL STRUCTURE AND MODEL ALIGNMENT ACROSS SCENE FACTORS

Human judgments vary selectively with scene geometry and physical parameters, and generative models reproduce some of these factor profiles while diverging on others. The largest reported generative profile error occurs for reflection color shifts.

J.2 HUMAN PERCEPTUAL STRUCTURE AND MODEL ALIGNMENT ACROSS SCENE FACTORS

  • For reflection color shifts, generative models scaled with discrepancy magnitude while humans already rated subtle shifts as markedly unnatural; generative median MSE was 1.292 versus encoder 1.037.For shape swaps, human ratings were consistently high across tested pairs, but generative scores varied by geometry; encoders aligned more closely for reflection shape swaps (MSE = 0.456 vs. generative 0.753).Source: §J.2 · pp. 40–41, Figure S15 · p. 42, Figure S17 · p. 44
  • Generative models matched human contrasts for shadow softness and object height, reflection floor roughness and object height, and light-direction surface and angle, but diverged on light-direction object height.For that height factor, human ratings were 0.850 versus 0.887, while generative scores were 1.070 versus 0.615.Source: §J.2 · p. 41, Figure S15 · p. 42, Figure S16 · p. 43
  • Generative models tracked human rotation sensitivity across shadows, reflections, and light direction, with median MSEs of 0.061, 0.324, and 0.170, respectively.Humans were sensitive to rotations of familiar shapes but near baseline for torus-knot rotations; encoders and FR-IQA metrics showed larger discrepancies.Source: §J.2 · p. 40, Figure S15 · p. 42, Figure S16 · p. 43

K EFFECTS OF MODEL CAPACITY ON SENSITIVITY AND HUMAN ALIGNMENT

Larger generators generally show stronger violation sensitivity and closer human alignment, but alignment scales selectively across conditions. Frozen encoders do not show the same consistent scaling pattern.

K EFFECTS OF MODEL CAPACITY ON SENSITIVITY AND HUMAN ALIGNMENT

  • Log parameter count correlated positively with generator violation sensitivity (Thatcher: Pearson r = .741, Spearman ρ = .685; Illumination: r = .831, ρ = .841) and human alignment (Thatcher: r = .627, ρ = .375; Illumination: r = .803, ρ = .796).The weaker Thatcher rank correlation indicates that alignment did not track size equally closely across models.Source: K EFFECTS OF MODEL CAPACITY ON… · p. 46, Figure S19 · p. 47
  • Across 12 frozen encoders, size was not consistently associated with sensitivity or alignment, particularly for Illumination.Encoder sensitivity was negatively associated with size on Thatcher (r = −.308, ρ = −.399), while encoder alignment was weakly associated with size on Illumination (r = −.110, ρ = −.098).Source: K EFFECTS OF MODEL CAPACITY ON… · p. 46, Figure S19 · p. 47
  • Sensitivity scaled positively with size across nearly all stimulus conditions, but alignment scaled with size for cast shadows and light direction, not surface reflections, and saturated early for upright faces.Reflection alignment showed no positive size association despite high human split-half reliability (r = .932), pointing to a divergence between model loss penalties and human reflection judgments.Source: K EFFECTS OF MODEL CAPACITY ON… · p. 46, Figure S20 · p. 48, Figure S21 · p. 49

L PUBLISHED GENERATION EVALUATIONS AND HUMAN ALIGNMENT

  • The authors also compared human alignment in their tasks with published generation evaluations to assess association beyond overall violation sensitivity.This passage states the purpose of the post-hoc comparison but gives no outcome.Source: L PUBLISHED GENERATION EVALUATIONS AND… · p. 50

L.1 EVALUATION CHOICE AND MODEL MATCHING

Published generation evaluations were compared with human alignment before and after sensitivity and size adjustment. The appendix also describes condition-averaged spatial loss-change maps across image and video models.

L.1 EVALUATION CHOICE AND MODEL MATCHING

  • Public text-to-image Elo scores were used for 11 image models, and VBench Quality scores for seven video models; models without corresponding public records were excluded.Published names did not guarantee identical checkpoint revisions or generation pipelines.Source: §L.1 · p. 50, Table S4 · p. 50

L.2 ASSOCIATIONS BEFORE AND AFTER SENSITIVITY ADJUSTMENT

  • After sensitivity adjustment, generation-evaluation and alignment associations remained positive for Illumination image models and both video tasks, but were weak for image models on Thatcher.The partial Spearman coefficients were .182 and .670 for image models, and .900 and .778 for video models, respectively.Source: §L.2 · p. 51, Figure S23 · p. 52

L.3 SENSITIVITY AFTER ACCOUNTING FOR HUMAN ALIGNMENT

  • After controlling for human alignment, residual associations between generation evaluations and violation sensitivity were weak across image and video models.The authors describe these observations as preliminary given the modest sample sizes.Source: §L.3 · p. 53, Figure S24 · p. 53

L.4 ADDITIONAL SIZE ADJUSTMENT AND INTERPRETATION

  • After controlling for sensitivity and scoring-backbone size, generation evaluations remained positively associated with human alignment for Illumination image models and both video tasks, while the image-Thatcher association remained weak.The authors report that controlling for parameter count did not attenuate these associations.Source: §L.4 · p. 54, Table S6 · p. 54

M SPATIAL LOSS-DIFFERENCE MAPS ACROSS GENERATIVE MODELS

  • Condition-averaged maps show paired modified-minus-original loss differences at native spatial positions, averaged over timesteps and noise draws, with video latent frames also averaged.For display, maps were cropped, resized to a common 96 × 96 grid, and shown with a shared symmetric color scale; warm colors indicate loss increases and cool colors decreases.Source: M SPATIAL LOSS-DIFFERENCE MAPS ACROSS… · p. 54, Figure S25 · p. 55, Figure S26 · p. 56

N TIMESTEP PROFILES AND PEAK LOCATIONS

Across conditions, sensitivity usually peaked later in the normalized denoising schedule than human alignment, with the separation varying by intervention. Additional controls found that alignment patterns were largely preserved under descriptive prompts and that the sensitivity–alignment association was not driven by shared sampling noise.

N TIMESTEP PROFILES AND PEAK LOCATIONS

  • The schedule coordinate maps stored timestep indices to [0, 1], from earliest, maximum-noise step to final, minimum-noise step; condition-specific sensitivity uses a task-wide pooled standard deviation.Peaks are signed maxima on each model’s discrete grid, without smoothing or interpolation.Source: N TIMESTEP PROFILES AND PEAK LOCATIONS · p. 58
  • Sensitivity peaked later than alignment in 20 of 25 models for upright faces (median shift ∆= +0.184) and 18 of 25 for inverted faces (median ∆= +0.192); for illumination it peaked later in 21 of 25 light-direction, 22 of 25 reflection, and 15 of 25 cast-shadow cases.The shift was largest for reflections and smaller for cast shadows.Source: N TIMESTEP PROFILES AND PEAK LOCATIONS · p. 58, Figure S28 · p. 58

O CONTROL FOR TEXT CONDITIONING

  • With descriptive prompts, alignment changes ranged from −.034 to .014 on Thatcher and from −.021 to .041 on Illumination, while model rankings correlated .967 and .983 with empty-prompt rankings.The prompts described stimulus content without mentioning the manipulation or judging naturalness.Source: O CONTROL FOR TEXT CONDITIONING · p. 61, Table S7 · p. 61, Table S8 · p. 61

P CONTROL FOR SHARED SAMPLING NOISE

  • Same-half and cross-half sensitivity–alignment associations differed by less than .001 in both tasks, indicating that shared sampling noise did not drive their relationship.The control used 19 models and split each model’s draws into fixed even and odd halves.Source: P CONTROL FOR SHARED SAMPLING NOISE · p. 62, Table S9 · p. 62

Related papers

Loading 2610.09928v1…