Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Explainability-in-Action: Enabling Expressive Manipulation and Tacit Understanding by Bending Diffusion Models in ComfyUI

Created by
  • Haebom

Author

Ahmed M. Abuzuraiq, Philippe Pasquier

Outline

This paper argues that explainable AI (XAI) can support artistic engagement, modifiability, and ongoing practice beyond transparency in creative contexts. While traditional refined datasets and human-scale model training can offer artists greater autonomy and control, large-scale generative models, such as text-to-image diffusion systems, often obscure this potential. This paper proposes that even large-scale models can be treated as creative materials if their internal structure is exposed and manipulable. We propose a technology-based explainability approach rooted in long-term, direct engagement, akin to Schön's "reflection-in-action," and demonstrate its application through model bending and inspection plugins integrated into ComfyUI's node-based interface. We demonstrate that by interactively manipulating different parts of a generative model, artists can develop intuition about how each component influences the output.

Takeaways, Limitations

Takeaways:
Presenting new possibilities for using large-scale generative models as creative materials.
Increased artistic control through exposure and manipulation of the model's internal structure.
Presentation and demonstration of a technology-based XAI approach based on “reflection in action”
Enhance interactive model manipulation and intuitive understanding through the ComfyUI plugin.
Limitations:
The presented approach is limited to a specific platform, ComfyUI. Further research is needed to determine its generalizability to other creation models and interfaces.
It requires a deep understanding of the internal structure of the model, which can be a high barrier to entry for artists.
Due to the complexity of large models, it can be difficult to fully understand the impact of all components.
Further validation of the generality and extensibility of the proposed plugin is needed.
👍