DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation

DreamBooth introduces a method for personalizing large text-to-image diffusion models so they can depict specific subjects, something standard models cannot do despite their high-quality synthesis. Given just a few images of a subject, the approach fine-tunes a pretrained model to bind a unique identifier to that subject. To retain diversity and the broader class, it introduces an autogenous class-specific prior preservation loss that leverages the semantic prior already embedded in the model.

Once the subject is embedded in the model's output domain, its unique identifier can be used to synthesize novel, photorealistic images of that subject in diverse scenes, poses, views, and lighting conditions absent from the reference images. The authors demonstrate the technique on subject recontextualization, text-guided view synthesis, and artistic rendering while preserving the subject's key features, and they provide a new dataset and evaluation protocol for this task of subject-driven generation.