LAION-5B: An open large-scale dataset for training next generation image-text models

Groundbreaking language-vision architectures such as CLIP and DALL-E demonstrated the utility of training on large amounts of noisy image-text data instead of expensive, accurately labeled data, and later models like ALIGN, BASIC, GLIDE, Flamingo, and Imagen made further improvements. However, studying the training and capabilities of such models requires datasets containing billions of image-text pairs, and until this work no dataset of that size had been made openly available. To address this and democratize research on large-scale multimodal models, the authors present LAION-5B, a dataset of 5.85 billion CLIP-filtered image-text pairs, of which 2.32 billion contain English-language text.

The authors show successful replication and fine-tuning of foundational models such as CLIP, GLIDE, and Stable Diffusion using the dataset, and discuss further experiments that an openly available dataset of this scale enables. They also provide several nearest-neighbor indices, an improved web interface for dataset exploration and subset generation, and detection scores for watermark, NSFW, and toxic content. By opening a dataset of this unprecedented scale, the work made large multimodal model research accessible to the broader community.