In this paper, we propose Attribute (Att) Adapter, a novel method to precisely control multiple continuous attributes (e.g., eye opening degree, car width) simultaneously in a pre-trained text-to-image diffusion model. Att-Adapter learns a single control adapter from a set of unpaired sample images and utilizes a decoupled cross-attention module to harmonize multiple domain attributes and text conditions. To mitigate overfitting, we additionally introduce a Conditional Variational Autoencoder (CVAE) to reflect various visual world characteristics. Experimental results show that Att-Adapter outperforms existing LoRA-based methods, demonstrating a wider control range and improved attribute separation performance. In addition, it can be trained without paired synthetic data and has excellent scalability for multiple attributes.
