SSTQ:面向隐私保护的子采样随机TurboQuant向量量化
SSTQ:Privacy-Preserving Vector Quantization via Subsampled Stochastic TurboQuant
SSTQ把向量量化做得更省位,联邦学习通信开销和隐私保护能兼得,比vqSGD那类老方法更有效。
SSTQ是面向联邦学习的隐私保护向量量化框架,结合了过完备等范数紧框架、坐标子采样和一维量化。与vqSGD等既有向量量化方法相比,SSTQ避免了维度相关的方差,达到最优均方误差缩放。
它用每个客户端仅ceil(log2 N)+b位实现这一效果,其中N=Θ(d)为框架大小。SSTQ含Flat Randomized Response和Metric-Aware Laplace两个变体,后者更适合高码本位宽设置。
其隐私感知码本目标将码本相关的均方误差缩放从O(4^b)降至O(2^b),并在CIFAR-10和Fashion-MNIST联邦学习任务上验证了效用与通信效率。
SSTQ:Privacy-Preserving Vector Quantization via Subsampled Stochastic TurboQuant
Achieving local differential privacy in distributed optimization while maintaining low communication cost remains challenging. Existing vector quantization methods, such as vqSGD, use high-dimensional geometric constructions but incur unfavorable dimension-dependent variance. In this work, we propose Subsampled Stochastic TurboQuant (SSTQ), a framework that combines overcomplete equal-norm tight frames, coordinate subsampling, and privacy-aware one-dimensional quantization. SSTQ includes two variants: a Flat Randomized Response version and a Metric-Aware Laplace version, the latter being better suited to higher codebook bit-width regimes. We show that SSTQ achieves optimal mean squared error scaling while using only $\lceil \log_2 N \rceil + b$ bits per client, where $N = Θ(d)$ is the frame size. We also derive a surrogate privacy-aware codebook objective that reduces the codebook-dependent MSE scaling from $O(4^b)$ to $O(2^b)$. Finally, we empirically evaluate SSTQ against established baselines on federated learning tasks using CIFAR-10 and Fashion-MNIST, demonstrating favorable utility and communication efficiency.
This report is categorized under AITOP's "Papers" desk. Key entities and technical topics under continuous tracking include SSTQ, TurboQuant, differential privacy, federated learning.
Source attribution: curated from arXiv cs.LG (@Adel Javanmard, David P. Woodruff, Vahab Mirrokni).