通过图像内在维度签名检测图像生成服务是否用了 LoRA 适配
Provenance Signatures for Low Rank Adaptations of Image Diffusion Models
有人担心在线服务偷偷套 LoRA 生成有害图片？这篇论文给了一套黑盒检验方法，不用碰模型内部就能分辨基础模型和适配版本。
arXiv 论文提出一种检测在线图像生成服务是否在基础扩散模型上套用 LoRA 适配的协议。与以往只在已知候选模型列表内做识别的工作不同，该研究针对开放世界场景：候选适配和替代模型集都未知。方法基于假设检验，用模型生成图像集合的内在维度构建黑盒签名，可区分基础模型、LoRA 适配以及完全不同的模型家族。论文还定义了多个模型家族与多个适配并存时的两类问题及对应威胁模型。
Provenance Signatures for Low Rank Adaptations of Image Diffusion Models
Diffusion models have revolutionized the generation of realistic artificial images. Efficient techniques such as Low Rank Adaptation (LoRA) can modify these models for particular use cases, making them more diverse and widely usable. At the same time, open base models may be adapted to deliver biased, harmful or illegal images in an online service. In this paper, we develop a protocol to detect when a service may be using an adaptation of an otherwise standard model. We describe the precise setup and threat model for this provenance question and define two different problems arising due the presence of multiple model families and multiple adaptations. In contrast to previous works that have considered model identification within lists of known candidate models, we consider the realistic open world scenario where the set of possible adaptations and alternative models are unknown. Using a hypothesis testing approach, our protocols can distinguish between the base model, a LoRA adaptation and other completely different families of models. For this test, we develop a black box signature of the models based only on intrinsic dimension of sets of images generated by the models.
