PaperlayerView exact revision on arXiv

Source-linked AI summary

LAION-5B: An open large-scale dataset for training next generation image-text models

Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, Jenia Jitsev

arXiv:2210.08402v1cs.CVcs.AIcs.LG

TL;DR

Large-scale language-vision research lacked openly available datasets with billions of image-text pairs. LAION-5B assembles and CLIP-filters over 5.8 billion public image-text examples, and subsets support CLIP models matching or approaching proprietary-model performance across diverse evaluations.

  • Problem

    Large-scale language-vision models rely on billion-scale image-text datasets, but none of these training datasets were publicly available.

    Source: §1 · p. 2

  • Method

    LAION-5B assembles images and associated HTML alt-text from Common Crawl and filters the collection with an existing CLIP model.

    Source: §1 · p. 2, §3 · p. 4

  • Results

    Models trained on LAION-400M match or come close to OpenAI’s CLIP models across zero-shot classification, distribution shifts, VTAB, retrieval, and fine-tuning.

    Source: §1 · p. 3

  • Takeaways & Limitations

    LAION-5B broadens access to large-scale multilingual training and research on language-vision models previously restricted to proprietary datasets.

    Source: §8 · p. 13

    LAION-5B may overlap with downstream test sets included in Common Crawl, potentially overstating models’ true generalization capabilities.

    Source: §6 · p. 10

Abstract

from arXiv · show

Groundbreaking language-vision architectures like CLIP and DALL-E proved the utility of training on large amounts of noisy image-text data, without relying on expensive accurate labels used in standard vision unimodal supervised learning. The resulting models showed capabilities of strong text-guided image generation and transfer to downstream tasks, while performing remarkably at zero-shot classification with noteworthy out-of-distribution robustness. Since then, large-scale language-vision models like ALIGN, BASIC, GLIDE, Flamingo and Imagen made further improvements. Studying the training and capabilities of such models requires datasets containing billions of image-text pairs. Until now, no datasets of this size have been made openly available for the broader research community. To address this problem and democratize research on large-scale multi-modal models, we present LAION-5B - a dataset consisting of 5.85 billion CLIP-filtered image-text pairs, of which 2.32B contain English language. We show successful replication and fine-tuning of foundational models like CLIP, GLIDE and Stable Diffusion using the dataset, and discuss further experiments enabled with an openly available dataset of this scale. Additionally we provide several nearest neighbor indices, an improved web-interface for dataset exploration and subset generation, and detection scores for watermark, NSFW, and toxic content detection. Announcement page https://laion.ai/laion-5b-a-new-era-of-open-large-scale-multi-modal-datasets/

1 Introduction

LAION-5B addresses the lack of openly available billion-scale image-text datasets by releasing 5.85 billion CLIP-filtered examples for multimodal research. The authors validate its suitability for training image-text models while emphasizing that the current release is an initial, research-only step requiring further auditing and refinement.

Motivation

  • Large-scale image-text datasets enable recent advances in zero-shot classification, distribution-shift robustness, and text-guided image generation, but the underlying datasets are largely unavailable to the research community.CLIP improved ImageNet top-1 accuracy from 11.5% to 76.2%, while datasets used by models such as CLIP and BASIC contained hundreds of millions or billions of pairs.Source: §1 · pp. 1–2

Contribution

  • LAION-5B releases over 5.8 billion public image-text examples, comprising 2.32 billion English, 2.26 billion multilingual, and 1.27 billion language-independent examples.The dataset is assembled from Common Crawl using an existing CLIP model for filtering.Source: §1 · p. 2
  • LAION-5B is more than 20 times larger than other public English image-text datasets.Table 1 compares the sizes of public and private image-text datasets.Source: Table 1 · p. 2

Validation

  • Models trained on the 400-million-example LAION-400M subset match or approach OpenAI CLIP across zero-shot classification, distribution shifts, VTAB, retrieval, and fine-tuning.OpenCLIP ViT-L/14 models trained on LAION-400M are described as the first open-source reproductions of OpenAI’s largest released CLIP models.Source: Figure 1 · p. 2, §1 · p. 3

Limitations and ethics

  • The release is an initial, unaudited data product intended only for academic research, with deployment discouraged until models’ behavior and biases are carefully investigated.The authors also release the software stack used to assemble LAION-5B and frame the dataset as a first step toward a broadly applicable multimodal pre-training resource.Source: §1 · pp. 2–3

2 Related Work

Prior work advanced vision-language learning through shared image-text representations, larger-scale multimodal training, and text-guided generation. Dataset efforts progressively increased scale and multilingual coverage while addressing annotation quality and correspondence limitations.

Vision-Language Models

  • CLIP [58] aligned images and text in a shared representation space through contrastive learning, enabling strong zero-shot classification and progress under distribution shifts [78] [84].Source: §2 · p. 3
  • Subsequent models increased training scale, introduced frozen-image-representation adaptation and in-context learning, combined contrastive and captioning objectives, and extended CLIP to navigation and visual question answering [28] [54] [94] [2] [43] [89] [32] [50] [72].Source: §2 · p. 3

Text-Guided Image Generation

  • Text-guided generation progressed from DALL-E’s multi-concept prompts [59] to systems that improved visual fidelity and text-prompt correspondence, including GLIDE, DALL-E 2, Imagen, Parti, and Stable Diffusion [52] [90].Source: §2 · p. 3

Image-Text Datasets

  • Human-annotated datasets offered high-quality labels but limited scale, while web-harvested datasets grew larger yet often lacked task-specific annotations or image-text correspondence; later cleaning and relaxed filtering produced datasets up to 1.8 billion pairs [36] [44] [79] [11] [26] [28].MS-COCO and Visual Genome contained 330K and 5M examples, respectively, whereas YFCC-100M contained about 99 million images and one million videos.Source: §2 · p. 4
  • Non-English image-text datasets were comparatively scarce: the largest multilingual dataset before LAION-5B had around 36 million samples, whereas LAION-5B provided roughly two orders of magnitude more multilingual samples for low-resource and multilingual research [75].Source: §2 · p. 4

Scaling Behavior

  • Scaling model, data, and compute improved generalization across computer vision and NLP, motivating large internal image datasets and community-accessible text corpora such as The Pile [18] and BigScience ROOTS.Source: §2 · p. 4

3 Collection Methodology

LAION-5B was assembled from Common Crawl by extracting alt-text image pairs, filtering them through distributed downloading and CLIP-based content screening, and retaining just under 6 billion examples. Safety handling emphasizes tagging potentially problematic content rather than claiming perfect removal, enabling user-controlled inclusion while documenting classifier limitations.

3 Collection Methodology

  • Common Crawl provides the source: monthly snapshots contain about 300 TiB across roughly 3 billion web pages, from which the pipeline extracts images and associated HTML alt-text.The dataset begins with a public web archive whose snapshots have been collected since 2008.Source: §3 · p. 4

3.1 Dataset Assembly Pipeline

  • The assembly pipeline combines distributed web-page filtering, asynchronous image-text downloading, and content filtering, with inclusion determined by distributed inference above a specified CLIP threshold.Image-text pairs are extracted from IMG tags with alt-text, language-detected, downloaded, and processed through the pipeline shown in Figure 2.Source: §3.1 · pp. 4–5, Figure 2 · p. 5
  • Removing short text, small images, malicious or redundant images, and low-similarity pairs eliminated around 90% of 50 billion images, leaving just short of 6 billion examples.The pipeline used ViT-B/32 CLIP models throughout, with cosine-similarity thresholds of 0.28 for English pairs and 0.26 for other languages.Source: §3.1 · pp. 5–6

3.2 Safety During Collection

  • Safety filtering tags potentially harmful or offensive content instead of removing it wholesale, because automated classifiers can miss harmful images, falsely remove others, and amplify social bias [76].Q16 and a specialized NSFW classifier based on CLIP embeddings identify and document inappropriate concepts; users can decide which tagged subsets to include.Source: §3.2 · p. 6

4 Dataset Composition

LAION-5B is released in three subsets totaling billions of image-text pairs, separated by detected language confidence. Apache Parquet metadata supports filtering and analysis, including NSFW and watermark scores.

Language subsets

  • LAION-5B comprises 2.32 billion English pairs, 2.26 billion pairs across over 100 other languages, and 1.27 billion samples with undetected language.The five most frequent multilingual languages are Russian (10.6%), French (7.4%), German (6.6%), Spanish (6.6%), and Chinese (6.3%).Source: §4 · pp. 6–7
  • Low-confidence language samples often depict products or places, with semantically clear captions that may include search-engine-optimization keywords or product tags.This characterization is based on visual inspection of a random subset.Source: §4 · p. 7

Metadata

  • Apache Parquet metadata provides identifiers, image URLs, text, dimensions, embedding cosine similarity, and NSFW and watermark detector outputs.The files are intended to support dataset filtering and analysis.Source: §4 · p. 7

Content filtering

  • 3% of images were detected as NSFW and can be filtered using the NSFW tag.Each metadata record also includes a watermark-detector score between 0 and 1.Source: §4 · p. 7

5 Experiments Validating LAION-5B

The experiments demonstrate LAION-5B’s utility for training and curating large-scale vision-language datasets, reproducing CLIP, improving zero-shot transfer with scale, and supporting generative-model training. Results include robustness comparable to original CLIP and stronger performance from larger data or model scale.

5.1 Usage Examples

  • LAION-5B enables curated subsets including LAION-High-Resolution, a 170M superresolution subset, and LAION-Aesthetic, a 120M subset selected by a CLIP-based linear estimator.LAION-5B and LAION-400M have also been used for facial representation learning and invasive species mitigation.Source: §5.1 · p. 7

5.2 Experiments on CLIP Reproduction

  • CLIP reproduction trained models of multiple sizes on LAION-400M, while LAION-5B subsets have also supported GLIDE fine-tuning and publicly available text-to-image generation models.The CLIP experiments used ViT-B/32, ViT-B/16, ViT-B/16+, and ViT-L/14; the generative-model reproduction is called LAIONIDE.Source: §5.2 · p. 8, §5.3 · p. 9

5.2.1 Zero-Shot Classification and Robustness Performance

  • LAION-trained CLIP models match original CLIP overall in robustness, while LAION-2B-en improves over LAION-400M across evaluated datasets for ViT-B/32 and ViT-L/14.On distribution shifts, original CLIP is better on ObjNet whereas LAION is better on INet-S; the comparison also includes VTAB+ averages over 35 tasks.Source: §5.2.1 · p. 9
  • VTAB+ accuracy improves with compute, and increasing either model or data scale improves zero-shot classification across downstream transfer targets.The compute-accuracy relationship is shown for models trained on LAION-400M and LAION-2B-en in Figure 4.Source: §5.2.1 · p. 9
  • 75.4% vs. 73.9%: at equal compute, ViT-L/14 trained on LAION-2B-en outperforms LAION-400M on ImageNet-1k zero-shot classification.The larger dataset improves performance when sufficient compute is available; the same effect also occurs for ViT-B/32.Source: §5.2.1 · p. 9

6 Technical Limitations

The section outlines potential technical limitations of LAION-5B as starting points for future dataset analysis and improvement. Key concerns include downstream test-set overlap, unreliable alt-text, and biases or noise introduced by CLIP-based filtering.

Data Overlap

  • Potential overlap between LAION-5B training data and downstream test sets could inflate accuracies and overstate generalization, although the authors do not consider it a serious threat.They note that OpenAI found few substantial performance differences from overlap [58], while benchmarks such as ObjectNet likely do not originate from Common Crawl; measuring overlap and improving de-duplication remain important future work.Source: §6 · p. 10

Other text sources

  • Alt-text can be SEO spam, keyword lists, incoherent, or corrupted, making annotations less informative or useless for training.Replacing corrupted captions with concatenated class names, as demonstrated by BASIC [54], may provide a meaningful alternative and support combining generated with natural-language captions.Source: §6 · pp. 10–11

Filtering with CLIP

  • CLIP filtering makes LAION-5B collection scalable with little human control but also transfers CLIP’s flaws and model biases into the dataset.The authors refer safety and ethics concerns to Appendix Sec. G.2.Source: §6 · p. 11
  • Filtering with the smaller CLIP ViT-B/32 may retain weakly connected pairs and remove high-quality pairs that stronger models could preserve.The authors hypothesize that CLIP ViT-L/14 would produce less noisy LAION datasets and propose testing ViT-L/14-filtered LAION-400M and LAION-5B in future work.Source: §6 · p. 11

7 Safety and Ethical Discussion

LAION-5B is presented as an open research artefact that enables broad investigation of model capabilities, biases, and safety risks otherwise obscured by closed datasets. The authors emphasize transparent, careful academic use, improved filtering, privacy considerations, and community-driven curation toward safer datasets.

Open investigation and safety

  • LAION-5B enables systematic study of how uncurated large-scale data affects model biases and safety, while supporting replication and development of automated curation methods.The authors argue that open datasets let the broader research community test both models and their training data for causes of undesired behavior.Source: §7 · p. 12

Filtering improvements

  • The dataset includes improved inappropriate-content tagging and a watermark filter intended to improve the safety and quality of text-to-image models.These measures were motivated by investigations of problems in the previously released LAION-400M dataset.Source: §7 · p. 12

Future curation

  • LAION-5B is a starting point for open, transparent dataset and training procedures, offering over 5.8 billion pairs for balancing data and selecting safe application-specific subsets.The authors encourage broad community participation in developing improved datasets rather than relying on closed-door processes.Source: §7 · p. 12

Responsible use

  • The authors recommend academic use only, careful investigation of downstream biases, and transparent reporting of newly detected content and model behavior.They also encourage users to apply the provided exploration tools and contribute findings to the dataset repository.Source: §7 · p. 12

Privacy

  • The paper notes privacy issues arising from Common Crawl links in LAION-5B and refers to Appendix Sec. G.1 for the measures undertaken to address them.The privacy discussion is explicitly separated from the broader safety and bias discussion.Source: §7 · p. 13

8 Conclusion · Appendix (LAION-5B: An open large-scale dataset for training next generation image-text models)

LAION-5B advances open language-vision research by providing over 5 billion image-text pairs, enabling stronger zero-shot transfer and robustness. Its subsets support competitive CLIP training, generative-model fine-tuning, multilingual research, and high-quality specialized datasets.

8 Conclusion

  • Over 5 billion image-text pairs further advance the scale of openly available datasets for training and studying state-of-the-art language-vision models.LAION-5B is described as a larger updated version of an openly available dataset.Source: §8 · p. 13
  • The dataset’s increased scale provides strong gains in zero-shot transfer and robustness.Source: §8 · p. 13
  • A LAION-5B subset trains state-of-the-art CLIP models across scales that match the zero-shot and robustness performance of original models trained on closed curated data.Source: §8 · p. 13
  • LAION-5B subsets can fine-tune generative models such as GLIDE, producing samples of good quality.Source: §8 · p. 13
  • The dataset broadens access to multilingual large-scale language-vision training and research previously restricted to proprietary datasets.Source: §8 · p. 13
  • Strict filtering for NSFW content, watermarks, resolution, and other criteria still yields datasets large enough to train or fine-tune strong specialized models.Source: §8 · p. 13

A Datasheet for LAION-5B dataset … C.5 Specialized NSFW image content tagging

This section presents LAION-5B as an open, large-scale collection of web image-text pairs, detailing its distributed construction, indexing, embeddings, distribution, maintenance, and anticipated uses. It also addresses the dataset’s safety, legal, and web-data limitations, including the development and evaluation of an open NSFW detector.

A.1 Motivation

  • LAION-5B was created as an open solution for training and studying very large multimodal models, expanding access beyond a few selected institutions.LAION presents the dataset as a non-profit effort to democratize access to large-scale open datasets and models.Source: §A.1 · p. 22

Q5 What do the instances that comprise the dataset represent (e.g., documents, pho-

Q7 Does the dataset contain all possible instances or is it a sample (not necessarily

  • The dataset is a 5.8-billion-pair subset of more than 50 billion Common Crawl images, retaining raw URLs and associated alt-text rather than hard class labels.Inclusion uses cosine similarity thresholds: above 0.28 for English and at least 0.26 for multilingual and unknown-language samples.Source: Q7 Does the dataset contain all… · p. 23, §A.3 · p. 25

Q14 Is the dataset self-contained, or does it link to or otherwise rely on external

  • Because samples rely on live web links, the authors cannot guarantee their continued existence or archive the dataset’s current image state.The provided img2dataset tool enables downloading the full dataset in about 7 days using 10 nodes, depending on bandwidth.Source: Q14 Is the dataset self-contained, or… · p. 24

Q15 Does the dataset contain data that might be considered confidential (e.g., data

  • The collection includes sexually explicit, racist, abusive, and other disturbing content; safety tags and boolean flags support filtering but are imperfect and unsuitable for guaranteeing a production-ready safe subset.Users are urged to apply the tags responsibly, while the dataset is intended for safety research and further curation of sensitive content.Source: Q15 Does the dataset contain data that… · p. 24, Q21 Any other comments? · p. 25

Q32 If consent was obtained, were the consenting individuals provided with a mecha-

Q41 What (other) tasks could the dataset be used for?

  • LAION-5B is intended for image and text representation learning, image-to-text generation, image captioning, multimodal tasks, safety research, and low-resource language research.The dataset has already supported CLIP reproduction, BLIP, GLIDE, Cloob, image captioning, facial representation learning, latent diffusion, and other systems.Source: §A.5 · p. 28, Q41 What (other) tasks could the… · p. 29

Q42 Is there anything about the composition of the dataset or the way it was collected

  • Because LAION-5B mirrors internet biases involving subpopulation depiction, violence, and nudity, it should not support decisions about people or military and surveillance tasks.The uncurated dataset is described as unsuitable for production and intended solely for research; safety tags are provided for nudity.Source: Q42 Is there anything about the… · p. 29

A.6 Distribution

  • The dataset is open-source, distributed through Hugging Face, released with LAION metadata under CC-BY-4.0, but LAION does not own the images’ or text’s copyrights.Hugging Face hosts metadata, The Eye hosts embeddings and backups, and LAION maintains distributed samples; LAION-5B will not be updated, though samples can be removed on request.Source: §A.6 · p. 29, Q49 Have any third parties imposed… · p. 30, §A.7 · p. 30

B Dataset Setup Procedure

  • The dataset setup combines distributed downloading, ViT-L/14 embedding computation, KNN indexing, and NSFW and watermark tagging across 5B image URL/text samples.The resulting 9TB of image embeddings were used to build a large PQ128 KNN index.Source: B Dataset Setup Procedure · p. 31, §C.3 · p. 32

C.1 Distributed img2dataset

  • img2dataset downloads, resizes, and stores images and captions in WebDataset format, processing 100 million URLs in 20 hours on one node and 5B samples in one week using 10 nodes.The tool enables users to obtain the whole dataset or smaller subsets.Source: §C.1 · pp. 31–32

C.2 Distributed CLIP inference

  • ViT-L/14 embeddings were computed in one week using 32 NVIDIA A100s, and the resulting embeddings were released for clustering, indexing, and linear inference.The larger model runs at 312 samples/s per GPU, compared with 1800 samples/s for ViT-B/32.Source: §C.2 · p. 32

C.5 Specialized NSFW image content tagging

  • 96,45% of true NSFW images were correctly detected as NSFW, while 7,96% of SFW images were incorrectly discarded as NSFW.The binary classifier grouped “drawing” and “neutral” as SFW, and “hentai”, “porn” and “sexy” as NSFW; evaluation used 1000 manually inspected images per category.Source: §C.5 · p. 33

C.6 Further inappropriate content tagging · C.7 Watermark and safety inference

The paper catalogs potentially inappropriate concepts in LAION-5B and provides user-facing safety and watermark filtering based on annotated datasets and inferred tags.

C.6 Further inappropriate content tagging

  • Q16 tagging identified frequent concepts spanning humans, pornography, weapons, violence, terror, murder, slavery, racism, and hate.Halloween-related imagery and art or media such as movies, games, and comics were also potentially tagged, depending on displayed content.Source: §C.6 · p. 33
  • Filtering further depends on the use case and users’ opinions.Source: §C.6 · p. 33

C.7 Watermark and safety inference

  • The watermark annotation criteria distinguish LAION-5B samples with watermarks from non-watermarked images.The paper illustrates these criteria with annotated examples in the watermark test set.Source: Figure 6 · p. 34
  • The watermark and safety system used manually annotated, distribution-representative test samples and a training set augmented with KNN-retrieved examples.Source: §C.7 · p. 34
  • Inference was performed with the embedding-reader module.Source: §C.7 · p. 34
  • Integrated safety tags filtered out almost all unsafe results in the user interface, supporting confidence that generative-model training would not produce unexpectedly unsafe images.Source: §C.7 · p. 34

D Dataset Samples and Statistics … E.2 Distributed Training and InfoNCE Loss

The section characterizes LAION-5B through representative samples and distribution statistics, then details CLIP reproduction training and a distributed InfoNCE implementation that reduces memory scaling. It reports low-confidence language classification for names, numbers, and short text, while multilingual coverage is concentrated among the ten most frequent languages.

D Dataset Samples and Statistics

E Further Experimental Details and Results on CLIP reproduction

  • The CLIP reproduction experiments use LAION 400M and 2B-en subsets and evaluate zero-shot classification with VTAB+ and retrieval.The section documents the experimental results and provides reproduction details for CLIP [58].Source: E Further Experimental Details and… · p. 36

E.1 Training Details

  • Distributed training uses PyTorch DDP on multiple NVIDIA A100 GPUs with InfoNCE as in [58], AdamW, warmup followed by cosine decay, and weight decay 0.2.All models use β1 = 0.9 and β2 = 0.98; architectural and resource details appear in Tables 3 and 4.Source: §E.1 · p. 36, Table 3 · p. 37, Table 4 · p. 37

E.2 Distributed Training and InfoNCE Loss

  • Each worker gathers image and text embeddings from other workers so global positive and negative pairs contribute correctly to the contrastive loss and gradients.This additional inter-worker communication enables the global batch to be handled in distributed InfoNCE training.Source: §E.2 · p. 36
  • O(nN) memory complexity replaces O(N^2) for distributed InfoNCE, reducing scaling overhead from gigabytes to megabytes.The implementation materializes two n × N matrices per worker instead of an N × N matrix; for N = 32768, the naive float32 matrix requires 8 GB.Source: §E.2 · pp. 36–37

E.3 Detailed Results & Further Analysis · F Overview of Experiments and Results on Generative Models

LAION models achieve strong transfer and retrieval performance, with benefits from larger data and model scales but notable weaknesses on adversarial, specialized, and structured tasks. The paper also surveys generative-model experiments using GLIDE and Stable Diffusion trained on LAION-5B subsets.

E.3 Detailed Results & Further Analysis

  • The evaluation reports zero-shot classification on VTAB+ and retrieval on Flickr30K and MSCOCO, using prompts and class names collected from prior works [58] [94].Table 5 describes the VTAB+ datasets, while Table 7 reports zero-shot top-1 accuracy and Tables 8–9 report retrieval results.Source: §E.3 · p. 37, Table 5 · p. 38
  • LAION-trained models achieve few-shot transfer performance similar to OpenAI CLIP, while performance increases with larger datasets and larger models across seven benchmark datasets.The evaluation covers ImageNet, Food101, Cars, CIFAR-10, CIFAR-100, DTD, and SUN397.Source: Figure 10 · p. 39, §E.3 · p. 39, Figure 11 · p. 40
  • ImageNet-A shows a 24.3% difference between CLIP WIT and LAION models on ViT-L/14, but the adversarial dataset design makes the generality of this discrepancy unclear.ImageNet-A images were selected to be adversarial for an ImageNet-1k-trained ResNet-50, and the paper does not explain why LAION models are worse.Source: §E.3 · p. 39
  • Diabetic Retinopathy accuracy varies widely and is strongly affected by class imbalance, while mean per-class recall reduces the CLIP–LAION gap to 2.1% but remains low overall.The best mean per-class recall is 25.4%, achieved by ViT-B/32 trained on LAION-400M; the majority class constitutes 74% of samples.Source: §E.3 · pp. 39–40
  • LAION models perform poorly on VTAB+ structured tasks involving counting, depth, or position/angle prediction, leaving improvement of these capabilities as an open research question.The tasks include CLEVR, DSPRITES, SmallNORB, DMLAB, and KITTI.Source: §E.3 · p. 40
  • Larger LAION datasets improve zero-shot ImageNet-1k transfer when training uses sufficient compute, consistently benefiting both ViT-B/32 and ViT-L/14 models.Experiments vary LAION-80M, LAION-400M, and LAION-2B with 3B, 13B, and 34B samples seen; LAION-2B is especially stronger than LAION-400M at 34B samples.Source: §E.3 · pp. 37–38, Figure 12 · p. 41
  • LAION models consistently outperform CLIP WIT on MSCOCO retrieval and generally match or exceed it on Flickr30K, including 59.3% versus 58.4% text retrieval and 42% versus 37.8% image retrieval at R@1.The Flickr30K exception is image retrieval with ViT-B/16, where CLIP WIT performs better.Source: §E.3 · p. 41

F Overview of Experiments and Results on Generative Models

  • Generative-model experiments train GLIDE and Stable Diffusion using subsets of LAION-5B.This section provides an overview of the training experiments performed with these two generative models.Source: F Overview of Experiments and Results… · p. 41

F.1 GLIDE … G.2 Potential Biases Induced by CLIP Filtering

The paper fine-tunes GLIDE and trains Stable Diffusion on LAION subsets, while discussing privacy risks and biases introduced by inaccessible CLIP training data and cosine-similarity filtering.

F.1 GLIDE

  • LAIONIDE reintroduces human-image generation by fine-tuning OpenAI’s filtered GLIDE 64-pixel base checkpoint on LAION-5B.The released GLIDE checkpoints removed hate symbols and humans, making them incapable of generating human imagery.Source: §F.1 · pp. 41–42
  • LAIONIDE-v2 uses roughly 25 million English LAION-2B pairs after filtering NSFW samples, extreme aspect ratios, low-resolution images, and captions containing listed slurs.Training used an 8xA100 pod and removed non-English samples; the slur filter was acknowledged as incomplete.Source: §F.1 · p. 42
  • LAIONIDE-v3 is fine-tuned on roughly 1 million pairs from MS-COCO, Visual Genome, Open Images, and LAION-5B, reducing watermark outputs during manual analysis.The mixture included LAION-5B [36] [44] [55].Source: §F.1 · p. 42

F.2 Stable Diffusion

  • Stable Diffusion is trained progressively on LAION-2B-en and higher-resolution or improved-aesthetics subsets, including a final stage with 10% text-conditioning dropout.The schedule comprises 237,000 steps at 256x256, followed by 194,000, 515,000, and 390,000 steps at 512x512.Source: §F.2 · p. 46
  • Stable Diffusion produces representative artistic and photorealistic samples, illustrated in Figures 14 and 15.The examples include an artistic painting prompt and a photorealistic jungle-river prompt.Source: §F.2 · p. 46, Figure 14 · p. 46, Figure 15 · p. 47

G.1 Privacy

  • Because LAION-5B links public web content, it may expose personal information such as faces, medical images, and other private material.CLIP retrieval can help users locate potentially personal data and its hosting location, while researchers can study related safety measures.Source: §G.1 · pp. 46–47
  • LAION supports privacy mitigation through host-provider removal requests, dataset-link blacklisting, and ongoing work on face obfuscation or generation-based anonymization [87] [48].Generation-based methods could preserve learned face representations without retaining private facial data in training sets.Source: §G.1 · p. 47

G.2 Potential Biases Induced by CLIP Filtering

  • CLIP’s inaccessible original training data prevents straightforward assessment of its biases, whereas LAION-5B enables more open auditing of contrastive image-text models.Cosine similarity is only a heuristic: it can prefer malignant over benign captions and may amplify biases because it misses semantic and contextual nuance [6].Source: §G.2 · p. 48

Related papers

Loading 2210.08402v1…