这篇论文给自回归图像生成的安全问题提供了一个不需要人工标注的自我改进方案，用模型自己判断不安全图片然后修复码本，很实用。
该论文针对自回归统一多模态模型在文本到图像生成中的安全性问题，提出迭代自我改进码本方法。方法利用模型自身理解与判断能力识别不安全生成图像，无需人工标注。通过构建有害空间并更新码本消除有害映射，再在无害空间内自适应微调码本以保持生成质量。实验表明，该方法在不依赖外部反馈情况下迭代提升模型安全性。
Safe Autoregressive Image Generation with Iterative Self-Improving Codebooks
Unlike diffusion-based models that operate in continuous latent spaces, autoregressive unified multimodal models produce images by sequentially predicting discretized visual tokens. These tokens are derived from a codebook that maps embeddings to quantized visual patterns. The language-like architecture enables unified multimodal models to effectively capture text conditional information for generation, making them promising for text-to-image tasks. This also raises an interesting question: how safe are the images generated in such an autoregressive way? In this work, we propose iterative self-improving codebooks for safe autoregressive generation. We leverage the understanding and judgment capabilities of the unified multimodal model itself to identify unsafe generated images without human annotation. Subsequently, the inherent representations in the codebook are fixed to eliminate harmful mappings. Our method comprises two steps: first, we use the unified model to identify unsafe generations and construct corresponding harmful and safe image-text pairs. These pairs are used to construct the Harmful Space and guide updates to the codebook, thereby eliminating harmful outputs. Second, we perform adaptive fine-tuning on the codebook within the harmless space using safe image-text pairs to ensure the quality of generated images. These two steps are repeated until no further improvement is observed, producing a safety-enhanced model codebook. Without additional external feedback, the safety of models is improved iteratively.