This page organizes papers related to artificial intelligence published around the world. This page is summarized using Google Gemini and is operated on a non-profit basis. The copyright of the paper belongs to the author and the relevant institution. When sharing, simply cite the source.
Causal-Adapter: Taming Text-to-Image Diffusion for Faithful Counterfactual Generation
Created by
Haebom
Author
Lei Tong, Zhihua Liu, Chaochao Lu, Dino Oglic, Tom Diethe, Philip Teare, Sotirios A. Tsaftaris, Chen Jin
Causal-Adapter: Counterfactual Image Generation with Causal Interventions
Outline
Causal-Adapter is a modular framework for counterfactual image generation using a fixed text-to-image diffusion backbone. This method enables causal intervention on target attributes without altering the core identity of the image, consistently propagating the influence to causal dependent variables. Unlike previous approaches that rely on prompt engineering, Causal-Adapter leverages structural causal modeling augmented by two attribute regularization strategies: text embeddings and aligned prompt injection for precise semantic control, and conditional token-contrast loss to separate attribute elements and reduce spurious correlations. Causal-Adapter achieves state-of-the-art performance on synthetic and real-world datasets, with up to a 91% MAE reduction on Pendulum for precise attribute control and an 87% FID reduction on ADNI for high-quality MRI image generation.
Takeaways, Limitations
•
Takeaways:
◦
We present a robust and generalizable approach for generating counterfactual images from fixed diffusion models.
◦
Achieve precise attribute control and strong identity preservation.
◦
It demonstrates state-of-the-art performance on synthetic and real-world datasets.
