想审计合成数据是否偷学了你的信息？这篇论文给出了一个轻量级方案，无需模型权限，只需输出和留出集就能揪出隐私泄露。
该论文提出一个可定制的实证审计框架，用于检测合成数据中的隐私泄露。框架区分“真泄露”（系统直接复制用户信息）与“幻影泄露”（偶然生成用户数据）。通过将数据分为训练集和留出集，并应用统计假设检验，可判断泄露是否超出零学习或差分隐私基线。该方法无需模型访问、无需插入蜜罐、无需训练参考模型，仅需合成输出和留出控制集。实验表明，它作为成员推理攻击，能提供比传统数据审计方法更紧的隐私泄露下限，且计算资源需求少几个数量级。
Phantoms and Disclosures: a Causal Framework for Auditing Synthetic Data
The rapid adoption of generative AI and Large Language Models (LLMs) has spurred interest in synthetic data as a privacy-preserving alternative to sensitive real-world datasets. However, generating high-utility synthetic data often carries the risk of memorizing and regurgitating private information from the training corpus. In this work, we present a customizable empirical auditing framework designed to detect and explain such data disclosures. Our framework introduces a mechanism to distinguish between "true disclosures"-where the system directly reproduces a user's information-and "phantom disclosures''-where the system incidentally generates a user's data. By partitioning input data into training and holdout sets and applying rigorous statistical hypothesis testing, we determine if observed disclosures are consistent with strict privacy baselines, such as zero-learning or specific Differential Privacy (DP) bounds. Crucially, this approach requires no model access, no canary insertion, and no reference model training -only the synthetic output and a held-out control set. We demonstrate that this framework effectively functions as a membership inference attack, providing empirical lower bounds on privacy leakage that are tighter than prior data-based auditing methods. Our approach is model-agnostic, applies to any synthetic data generation mechanism, and requires orders of magnitude fewer computational resources than shadow-model or canary-based alternatives.