[공지사항]을 빙자한 안부와 근황 
Show more

Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Generating Synthetic Data via Augmentations for Improved Facial Resemblance in DreamBooth and InstantID

Created by
  • Haebom

Author

Koray Ulusan, Benjamin Kiefer

Outline

This paper evaluates the impact of augmentation strategies for maintaining facial similarity in Stable Diffusion personalization for generating professional-quality portraits from amateur photographs. For two personalization methods, DreamBooth and InstantID, we compare and analyze the conventional augmentation (left-right flipping, cropping, and color adjustment) with the generative augmentation using synthetic images of InstantID. We quantitatively evaluate facial similarity using SDXL and a novel FaceNet-based FaceDistance metric. The experimental results show that the conventional augmentation can produce artifacts that hinder the identification of people, whereas InstantID improves fidelity when used in a balanced manner with real images to avoid overfitting. A user study with 97 participants confirms the difference between the preference for the refined appearance of InstantID and the accurate identification of people of DreamBooth. The results of this study provide insights into effective augmentation strategies for personalized text-to-image generation.

Takeaways, Limitations

Takeaways:
We demonstrate that a generative augmentation strategy using InstantID provides higher photo realism and user preference in personalized portrait generation than DreamBooth.
Contributes to the qualitative improvement of personalized image creation by suggesting the shortcomings of existing augmentation strategies and appropriate utilization of generative augmentation strategies.
We present a novel method to quantitatively evaluate face similarity through a new FaceDistance metric.
Limitations:
The number of user study participants may be limited (97 participants).
Reliance on a specific face recognition model (FaceNet) may limit the generalizability of the results.
Further research is needed into other personalization methods beyond InstantID and DreamBooth.
Further research is needed to determine generalizability across different types of amateur photography.
👍