Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Style Transfer to Calvin and Hobbes comics using Stable Diffusion

Created by
  • Haebom

Author

Asvin Kumar Venkataramanan, Sloke Shrestha, Sundar Sripada Venugopalaswamy Sriraman

Outline

This project report summarizes the process of fine-tuning the Stable Diffusion model using the Calvin and Hobbes comics dataset. The goal is to perform style transfer, transforming an arbitrary input image into the Calvin and Hobbes comics style. For efficient fine-tuning, we trained stable-diffusion-v1.5 using Low Rank Adaptation (LoRA), and the diffusion process is handled by a Variational Autoencoder (VAE) in U-net. Considering the training time and input data quality, the results are visually appealing.

Takeaways, Limitations

Takeaways: We present an efficient fine-tuning method for a stable diffusion model using LoRA, demonstrating that it can achieve good style transfer results even with limited datasets and training times. We also present a practical approach for specific style transfer tasks, such as the Calvin and Hobbes comic book style.
Limitations: Detailed descriptions of the size and quality of the dataset used are insufficient. Since objective performance evaluation metrics are not provided, the results are solely based on qualitative assessments. Validation of generalization performance on images of different styles is lacking. Due to the nature of the report, detailed technical details and information on experimental setup are limited.
👍