为艺术拆解扩散模型:交互式模型弯曲与实践可解释性

Unboxing Diffusion Models for the Arts: Interactive Model Bending and Practice-Based Explainability

精选理由

这篇论文告诉你,艺术家也能像玩材料一样玩扩散模型内部。他们在ComfyUI做了交互工具,能直接操纵Stable Diffusion的层,看到具体变化。不是理论说教,是真能动手调的。

AI 摘要

论文提出一种面向创意实践的可解释AI方法,让艺术家能检查、修改和调试扩散模型内部结构。作者在ComfyUI的节点工作流中集成了交互式层选择与干预控件,实现模型弯曲。通过对Stable Diffusion 1.5进行定性与定量分析,发现操纵不同组件(如UNet层)会产生相对一致的视觉效应。该方法帮助艺术家建立层级的直觉,理解模型各部分如何影响生成图像。

原文 · arXiv cs.LG

Unboxing Diffusion Models for the Arts: Interactive Model Bending and Practice-Based Explainability

Explainable AI (XAI) in creative practice can be less about technocentric explanation and more about enabling artists to inspect modify and debug models as part of making Yet largescale texttoimage diffusion systems are typically presented as opaque endtoend tools limiting this kind of material engagement We argue that even large models can function as creative materials when their internal structure is made visible and manipulable To support this we propose a handson approach to explainability centred on experimentation and intervention We instantiate this approach with a model bending and an interactive (inspection) interface integrated into ComfyUIs nodebased workflow including interactive layer selection and intervention controls Through qualitative and quantitative analysis of bending interventions in Stable Diffusion 15 we show how manipulating specific components of a diffusion pipeline produces relatively consistent families of visual effects allowing artists to build practical layerlevel intuition about how different parts of the model shape generated images