PaperlayerView exact revision on arXiv

Source-linked AI summary

Evaluating Data Attribution for Text-to-Image Models

Sheng-Yu Wang, Alexei A. Efros, Jun-Yan Zhu, Richard Zhang

arXiv:2306.09345v2cs.CVcs.LG

TL;DR

Data attribution asks which training images most influenced a generated image, but existing approaches are difficult to scale to large generative models. This paper builds an exemplar-influenced dataset using customization, evaluates and tunes retrieval features, and reports generalization to larger image sets alongside soft attribution scores.

  • Problem

    Identifying which training images most shaped a generated image is difficult, and influence methods are hard to scale to generative models.

    Source: §1 · pp. 1–2

  • Method

    The authors tune pretrained generators toward exemplars to create an attribution dataset, then evaluate and train retrieval features on it.

    Source: §1 · p. 2

  • Results

    The exemplar-based method generalizes to sets of 100–1000 unrelated images, and trained features improve attribution on MSCOCO models in most cases.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    Calibrated retrieval scores can represent uncertain influence across multiple candidate training images.

    Source: §1 · p. 2, §4.2 · p. 6

    The method analyzes full images rather than compositional attribution, arbitrary images can be difficult to customize, and pretraining images introduce label noise.

    Source: §6 · p. 9

Abstract

from arXiv · show

While large text-to-image models are able to synthesize "novel" images, these images are necessarily a reflection of the training data. The problem of data attribution in such models -- which of the images in the training set are most responsible for the appearance of a given generated image -- is a difficult yet important one. As an initial step toward this problem, we evaluate attribution through "customization" methods, which tune an existing large-scale model toward a given exemplar object or style. Our key insight is that this allows us to efficiently create synthetic images that are computationally influenced by the exemplar by construction. With our new dataset of such exemplar-influenced images, we are able to evaluate various data attribution algorithms and different possible feature spaces. Furthermore, by training on our dataset, we can tune standard models, such as DINO, CLIP, and ViT, toward the attribution problem. Even though the procedure is tuned towards small exemplar sets, we show generalization to larger sets. Finally, by taking into account the inherent uncertainty of the problem, we can assign soft attribution scores over a set of training images.

1. Introduction

Data attribution asks which training images most shaped a synthesized output, but methods for estimating influence are difficult to scale to generative models. The paper uses exemplar customization to construct an attribution dataset, evaluate retrieval features, and train attribution models that generalize beyond small exemplar sets.

1. Introduction

  • Attribution methods such as influence functions and subset-based analysis are difficult to scale from classifiers to generative models with billions of images and parameters.Training the many models needed to test image-removal effects is computationally infeasible for generation.Source: §1 · pp. 1–2
  • Predefined feature spaces enable retrieval, but features such as CLIP may not suit attribution, motivating objective evaluation of candidate spaces.Visual similarity alone does not establish data influence.Source: §1 · p. 2
  • Customization produces synthesized images computationally influenced by an exemplar, providing noisy but informative ground-truth attribution data.The dataset pairs each exemplar with images synthesized after tuning a pretrained model toward it.Source: §1 · p. 2
  • The dataset evaluates candidate retrieval features and supports contrastive tuning of feature spaces for attribution.It also estimates soft influence scores over multiple candidate training images using a thresholded softmax over retrieval scores.Source: §1 · p. 2
  • The exemplar-based procedure generalizes to tuning on larger sets of 100–1000 unrelated images.The paper presents this as evidence of applicability to the broader data-attribution problem.Source: §1 · p. 2
  • The paper summarizes its contributions as an exemplar-influenced dataset, feature evaluation and improvement, and soft attribution scores over training images.The authors also release code, a model, and the dataset.Source: §1 · p. 2

2. Related Work

Prior work studies model identification, training-data membership, influence estimation, and approximate-copy detection, while this paper addresses attribution of specific training examples in generative models. It adapts ideas from influence and representation-learning research to a setting where repeated generative-model training is prohibitively expensive.

Related work

  • Model attribution identifies the model that produced a synthetic image, whereas this work assumes the model is known and seeks the specific training data that enabled it.Membership inference asks whether an image is in a model’s training set.Source: §2 · p. 2
  • Shapley-value and loss-tracing approaches assess datapoint contributions in discriminative models, unlike this work’s focus on generative models.Feldman and Zhang [17] train on random subsets; Pruthi et al. [52] trace training-loss changes.Source: §2 · pp. 2–3
  • Repeatedly training large generative models to estimate each training point’s value is prohibitively expensive, so the paper simulates influence by adding exemplar influence.This contrasts with post hoc analysis of training influences.Source: §2 · p. 3
  • Replication detection seeks approximate copies, where influence may be near 100% for one image; this paper instead considers influence across the training set in general cases.The cited replication-detection works include [68] [6].Source: §2 · p. 3
  • Self-supervised representations support downstream recognition, and this paper evaluates their suitability for attribution and improves them with contrastive learning.The related representation-learning works include [53] [8] [24] [71] [10] [43].Source: §2 · p. 3

3. Creating an attribution dataset

The paper constructs an attribution dataset by tuning pretrained generators toward exemplar concepts and sampling images from the resulting models. It covers object and artistic-style concepts, using curated sources and prompt strategies to create distinct training and testing distributions.

3.1. “Add-one-in” training

  • Custom Diffusion fine-tunes a pretrained generator on an added concept, enabling tractable model collection instead of training a new generator for every concept.The method takes about 6 minutes and uses 75 MB of storage per model.Source: §3.1 · p. 3

3. Creating an attribution dataset

  • The dataset comprises more than 18,000 customized models and over 4 million samples across object and artistic-style sets, with distinct training and testing distributions.The authors vary prompts through ChatGPT and procedural generation.Source: Table 1 · p. 4

3.2. Object-centric models

  • Object-centric models use one ImageNet exemplar per model, with curated classes and ChatGPT or procedural prompts spanning varied scenarios and visual mediums.The collection includes 6,930 images from 693 classes; classes with unfaithful customized samples are removed.Source: §3.2 · p. 4

3.3. Artistic-style models

  • Artistic-style models are tuned on grouped BAM-FG images or Artchive artist collections, then sampled using ChatGPT captions and procedural prompts for varied objects.BAM-FG models are split for training, validation, and testing; Artchive models and held-out prompts support out-of-distribution testing.Source: §3.3 · p. 5

4. Evaluating and training for attribution

The section frames attribution as reversing the process that pairs exemplars with customized synthetic images: learn features to retrieve the influencing exemplars, then calibrate similarities into soft probabilities.

4. Evaluating and training for attribution

  • The section reverses the exemplar-to-synthesis process by predicting the influencing exemplar set from a generated image.The setup uses paired exemplar training images and influenced synthetic images.Source: §4 · p. 5

4.1. Evaluating existing features

  • Existing feature spaces provide retrieval baselines, but visual similarity is not equivalent to attributing data influence.The section evaluates candidate feature spaces with standard information-retrieval protocols; these features perform above chance.Source: §4.1 · p. 5

4.2. Learning features for attribution

  • The learned feature extractor retrieves exemplar training images from synthesized outputs, using paired views of training and synthetic images to train the attribution mapping.The contrastive objective uses other exemplar images as negatives rather than LAION images, to avoid learning to classify datasets; affine mappings performed best, and regularization addresses overfitting.Source: §4.2 · pp. 5–6
  • A calibrated softmax maps learned similarities to influence probabilities, with ReLU thresholding able to assign zero probability to low-influence images.Calibration subtracts the maximum similarity before applying a learned-temperature exponential, learned-offset ReLU, and normalization.Source: §4.2 · p. 6

5. Experiments

Training attribution features on the exemplar dataset improves performance across object-centric and artistic-style tasks, with gains across prompt types and domains. The evaluation also indicates transfer toward larger-set attribution, though a gap remains at the largest tested set.

Feature encoders and finetuning

  • ImageNet-pretrained DINO, ViT, and MoCo perform better on objects, while general encoders outperform specialized ALADIN and SSCD features, including for artistic styles.The authors attribute the object-task advantage to the smaller domain gap; SSCD is less suited to attribution when synthesized images vary substantially.Source: §5 · pp. 6–7
  • Combining object-centric and artistic-style training yields consistent gains on both axes; all three CLIP variants and object-centric DINO lie on the Pareto front.The figure compares artistic-style performance against object-centric performance and reports gains from training on the attribution dataset.Source: Figure 5 · p. 7, §5 · p. 7

Prompt types

  • Object-centric attribution works better for GPT-prompted than media-prompted images, while style-centric attribution is challenging across prompt types and finetuning improves pretrained features.Media prompts can introduce stylistic variation that makes attribution ambiguous; the figure reports gains across prompt types after training on the dataset.Source: §5 · p. 7, Figure 5 · p. 7

Out-of-domain tests

  • Learning on the dataset consistently improves both in-domain and out-of-domain tests, including unseen ImageNet classes and Artchive styles.Figure 6 plots performance on ImageNet-Seen and BAM-FG against performance on ImageNet-Unseen and Artchive.Source: Figure 6 · p. 7, §5 · p. 8

Ablation studies

  • Regularization alleviates overfitting, linear mapping performs best, and augmentation strength has only a minor effect on performance.Mild augmentation performs slightly better than strong augmentation.Source: Figure 7 · p. 7, §5 · p. 8

Soft influence scores

  • Calibrating the softmax temperature is necessary: at the default temperature, influence-score variances are below 0.0003% across training data.After calibration, related concepts receive higher influence scores.Source: §5 · p. 8

Towards general data attribution

  • For MSCOCO sets of 1, 10, 100, and 1,000 images, add-one-in benchmark performance generally correlates with MSCOCO performance, but correlation is less prominent at 1,000.Features tuned on object-centric or object-plus-style data also improve MSCOCO attribution; the authors note remaining headroom and a gap to general attribution.Source: §5 · pp. 8–9, Figure 10 · p. 9

Qualitative results on Stable Diffusion

  • On Stable Diffusion queries, tuned CLIP assigns high attribution scores to related LAION-400M concepts, with scores shared across similar images.The qualitative evaluation runs attribution over the full LAION-400M set.Source: Figure 9 · p. 8, §5 · p. 9

6. Discussion and Limitations

The work presents benchmarking as a first step toward understanding training-image attribution, which could support creator incentives. Its approach has constraints in compositional attribution, image customization, and the relationship between exemplars and large-scale training data.

Discussion

  • Training-image attribution could create opportunities to incentivize and reward creators for providing data.The paper positions its benchmarks as a step toward understanding the training process and validating future attribution methods.Source: §6 · p. 9

Limitations

  • The method evaluates full images rather than compositional attribution, and customization is difficult for arbitrary images, including some LAION images with unusual prompts.Pretraining images also influence outputs, creating label noise, though customization upweights the exemplar and supports aggregate attribution.Source: §6 · p. 9
  • Customization training leaves a domain gap to large-scale attribution, including in exemplar-image distributions and selected prompts, and requires balancing exemplar influence against sample diversity.The paper follows Custom Diffusion best practices for that tradeoff.Source: §6 · p. 9

A. Dataset Collection Details

The appendix details how the exemplar dataset and prompts are assembled, then illustrates why softmax temperature calibration makes influence scores more distinctive.

A. Dataset Collection Details

  • The dataset uses Custom Diffusion settings from Kumari et al. [36], omits regularization datasets for faster convergence, and early-stops each model at 125 iterations.The authors say regularization is unnecessary because dataset prompts relate to the input image.Source: A. Dataset Collection Details · p. 12
  • The 693 ImageNet classes are grouped into 55 broad categories, with category names chosen to be more natural for training and prompting than fine-grained labels.Object-centric models use 25 manually selected captions per category and replace category mentions with the tuned-concept token V*<category>.Source: A. Dataset Collection Details · p. 12
  • Style prompts include 50 manually selected painting captions, while BAM-FG prompts add 40 object variations using objects selected from a collection of 50.Prompts use the tuned style token V*art.Source: A. Dataset Collection Details · p. 13
  • Training and testing use different prompts to create an out-of-distribution test set, and prompt lists are supplied in separate files by model and prompt type.The appendix also points to visualizations of object-centric and artistic-style dataset samples.Source: Table 2 · p. 12, A. Dataset Collection Details · p. 13

B.1. Visualizing the calibrated influence scores.

  • Because cosine similarities occupy a small range, an uncalibrated softmax spreads influence scores evenly; temperature calibration makes top-attributed images more distinctive.Figure 11 compares score distributions before and after calibration for tuned DINO and CLIP features.Source: §B.1 · p. 13, Figure 11 · p. 13

B.2. Ablation Studies

The ablations favor a well-regularized linear mapper: higher-capacity alternatives overfit, while stronger augmentation is unnecessary. Regularization preserves pretrained feature properties and improves generalization.

Regularization

  • Regularization reduces significant overfitting and preserves pretrained feature properties, resulting in better generalization.The regularizer encourages the linear mapping to be a rigid transform.Source: §B.2 · p. 13

Mapping functions

  • A well-regularized linear mapping provides enough capacity to improve performance without the overfitting of a 2-layer MLP; channel-wise multiplication is insufficient.The MLP generalizes much worse on the test set, while channel-wise multiplication cannot improve performance.Source: §B.2 · p. 13

Copy detection

  • For copy detection, attribution scores drop significantly from similar training images to other images, with multiple LAION images resembling the query.The figure identifies training-image matches reported by Somepalli et al. [68] in red.Source: Figure 12 · p. 14

Data augmentations

  • Mild and strong augmentations perform similarly, with mild augmentation slightly favorable; strong augmentation is not necessary.The authors use mild augmentation in other experiments because the feature extractor is pretrained and the learned mapping is light.Source: §B.2 · p. 14

B.3. Additional Stable Diffusion results.

In tests of memorized Stable Diffusion samples, the method assigns high attribution scores to matched training images. Similar training images also receive higher scores than other images, and lower-ranked retrieved images often have zero scores.

Copy detection for Stable Diffusion

  • The method assigns high attribution scores to matched training images in tests of memorized Stable Diffusion samples.The tests add matches reported by Somepalli et al. [68] to a 1M random LAION subset.Source: §B.3 · p. 14
  • Attribution scores drop significantly between training images similar to a memorized sample and other images.The authors observe multiple training images similar to the memorized samples.Source: §B.3 · p. 14

Stable Diffusion attribution

  • For Stable Diffusion queries, the authors retrieve the top 10,000 LAION-400M images, and lower-ranked images in that subset often already have zero attribution scores.Source: §B.3 · p. 14

B.4. Additional Custom diffusion results.

Additional evaluations report consistent trends across metrics and show that tuning on the authors’ dataset outperforms a baseline trained without customization. A Stable Diffusion feature baseline does not outperform the evaluated features.

B.4. Additional Custom diffusion results

  • Recall@1, Recall@100, and mAP evaluations show the same general trends and method rankings as Recall@10 across in-domain and out-of-domain tests.The section also provides additional qualitative comparisons of pretrained and fine-tuned features.Source: §B.4 · p. 14

B.5. Additional Baselines

  • Stable Diffusion features score 0.279 versus DINO 0.553 and CLIP 0.342 at object-centric Recall@10; style-centric scores are 0.202, 0.199, and 0.201, respectively.The Stable Diffusion baseline also incorporates captions, while the compared feature spaces analyze images.Source: §B.5 · p. 14
  • For Recall@10, tuning DINO and CLIP features on the authors’ dataset improves vanilla-feature scores from 0.376 to 0.406 and 0.271 to 0.363, respectively, beating same-size-subset baselines at 0.393 and 0.359.The baseline features are tuned on a subset of the authors’ dataset of the same size.Source: §B.5 · p. 15

C.1. Training details for feature mappers

The feature-mapper training details specify mapper initialization, optimization, sampling, and image augmentation. A separate Stable Diffusion experiment fine-tunes models on MSCOCO subsets of varying sizes.

C.1. Training details for feature mappers

  • The mapper starts from an identity weight matrix with zero bias and is trained with Adam, an initial learning rate of 10^-3, and cosine decay.Batch size is 1024 except for ViT and ALADIN, which use 512 due to memory constraints.Source: §C.1 · p. 15
  • Each model is trained for 100 epochs, sampling one attribution pair per model per epoch and choosing training images, prompt types, and synthesized images randomly.Each minibatch samples B different models.Source: §C.1 · p. 15
  • Training randomly flips, resizes, and crops images to the encoder’s input size; testing uses center crops.Input size and normalization match each base feature encoder.Source: §C.1 · p. 15

C.2. Details for fine-tuning on MSCOCO

  • Stable Diffusion is fine-tuned on random MSCOCO subsets of 1, 10, 100, or 1000 images, with four random subsets and models per size.The experiment tunes cross-attention weights using MSCOCO captions and generates four samples per hold-out caption, except for models trained on 1000 images, which use one.Source: §C.2 · p. 15

C.3. Soft influence score implementation

The appendix implements calibrated soft influence scores from feature similarities, using top-match truncation and smoothing to make calibration practical. It also adds attribution examples and checks that additional metrics show similar trends and rankings.

C.3. Soft influence score implementation

  • Influence scores come from similarities between synthesized and training images, converted to percentage assignments with calibrated temperature and threshold.The inference pipeline uses both a base feature extractor and the learned embedding.Source: §C.3 · p. 15, Figure 13 · p. 15
  • Calibration retains the top 100,000 similarities, or 10%, and uses a smooth softplus approximation during training to stabilize optimization.The remaining similarities are discarded because they are expected to be compressed to zero by ReLU.Source: §C.3 · p. 15
  • For retrieval, training excludes LAION to avoid learning a LAION classifier, while influence scores are tuned on LAION without changing learned feature similarities..Source: §C.3 · p. 16

D. Change log.

  • The camera-ready release updates Figures 9 and 14 with attribution results on the full LAION-400M dataset and corrects Figure 11.Figure 14 shows generated queries alongside top attributed training images, with calibrated influence percentages.Source: D. Change log · p. 16, Figure 14 · p. 17
  • Fine-tuning improves the ranking and influence scores of exemplar training images in qualitative comparisons.The added examples cover object-centric and artistic-style models with GPT-generated and procedurally generated prompts.Source: Figure 15 · p. 18
  • Additional metrics show similar trends and rankings across methods and test cases.The visualization adds mAP, Recall@5, and Recall@100.Source: Figure 16 · p. 19
Loading 2306.09345v2…