SSTQ把向量量化做得更省位,联邦学习通信开销和隐私保护能兼得,比vqSGD那类老方法更有效。
SSTQ是面向联邦学习的隐私保护向量量化框架,结合了过完备等范数紧框架、坐标子采样和一维量化。与vqSGD等既有向量量化方法相比,SSTQ避免了维度相关的方差,达到最优均方误差缩放。它用每个客户端仅ceil(log2 N)+b位实现这一效果,其中N=Θ(d)为框架大小。SSTQ含Flat Randomized Response和Metric-Aware Laplace两个变体,后者更适合高码本位宽设置。其隐私感知码本目标将码本相关的均方误差缩放从O(4^b)降至O(2^b),并在CIFAR-10和Fashion-MNIST联邦学习任务上验证了效用与通信效率。
SSTQ:Privacy-Preserving Vector Quantization via Subsampled Stochastic TurboQuant
Achieving local differential privacy in distributed optimization while maintaining low communication cost remains challenging. Existing vector quantization methods, such as vqSGD, use high-dimensional geometric constructions but incur unfavorable dimension-dependent variance. In this work, we propose Subsampled Stochastic TurboQuant (SSTQ), a framework that combines overcomplete equal-norm tight frames, coordinate subsampling, and privacy-aware one-dimensional quantization. SSTQ includes two variants: a Flat Randomized Response version and a Metric-Aware Laplace version, the latter being better suited to higher codebook bit-width regimes. We show that SSTQ achieves optimal mean squared error scaling while using only $\lceil \log_2 N \rceil + b$ bits per client, where $N = Θ(d)$ is the frame size. We also derive a surrogate privacy-aware codebook objective that reduces the codebook-dependent MSE scaling from $O(4^b)$ to $O(2^b)$. Finally, we empirically evaluate SSTQ against established baselines on federated learning tasks using CIFAR-10 and Fashion-MNIST, demonstrating favorable utility and communication efficiency.