用虚拟点隐私化批次大小，DP-SGD 在截断批次下仍保隐私保证
Counting on Dummies: Privacy Amplification with Observable Counts and Truncated Batches
训练用 DP-SGD 时批次大小一泄露隐私就崩？这篇 arXiv 论文用随机 dummy points 解决截断和泄露问题，还比 2025 年 Ganesh 的直接核算更省。
arXiv 论文《Counting on Dummies》针对 DP-SGD 训练中 Poisson 采样和随机分配方案批次大小可变、且在分布式环境可能泄露的问题，提出用随机数量的 dummy points 虚拟点机制来隐私化批次大小。该机制在批次超过上限 s 被截断时仍能维持隐私保证，而此前已知截断会显著增加隐私损失。论文为 Poisson 采样和随机分配两种方案给出了一般性分析，支持较高效的隐私核算，并通过子采样使期望虚拟点数量渐近小于直接加非负噪声所需的规模。在截断批次的 Poisson 采样场景下，实证结果显示其隐私-效率权衡优于 Ganesh 在 2025 年给出的直接截断核算方法。
Counting on Dummies: Privacy Amplification with Observable Counts and Truncated Batches
In many practical deployments of Differentially Private Stochastic Gradient Descent (DP-SGD), the server processes data in batches. Randomly sampling these batches allows for better privacy bounds via privacy amplification results. Most sampling schemes with provable privacy amplification, including Poisson sampling and random allocation, result in variable batch sizes. The privacy guarantee of these schemes requires that the batch sizes are not revealed. In distributed settings, the server performing the aggregation or any party observing the communication traffic may learn these batch sizes. Furthermore, in both centralized and distributed settings, memory and computation constraints may require each batch to be of some maximum size $s$, necessitating truncation of the batch when it exceeds this limit. It is known that this truncation can result in a significant increase in privacy loss. In this work, we address both of these challenges via a simple mechanism that uses a random number of \emph{dummy points} to privatize the batch size and maintain the privacy guarantee even under truncation. We demonstrate that our mechanism is easy to implement in a distributed setting, and provide a general analysis of this mechanism for both Poisson sampling and random allocation. Our analysis is designed to support efficient and relatively tight privacy accounting. It also effectively amplifies the privacy of the count releases by subsampling, making the expected number of dummy points asymptotically smaller than what is needed to directly privatize the (sensitivity-one) batch sizes with non-negative noise. For Poisson sampling with truncated batches, we demonstrate empirically that our mechanism achieves better privacy--efficiency tradeoffs than the direct accounting for truncation recently given by (Ganesh, 2025).