想提升句向量对微小改动的抵抗力？这个CDAE方法比SimCSE更扛扰动，代码开源，可以试试。
该论文提出轻量级对比去噪自编码器CDAE，通过联合优化对比学习和重构目标来精炼BERT嵌入，使其对同义替换、掩码和词丢弃等语义保持扰动不敏感。实验在多种扰动强度下对比了原始BERT和SimCSE，CDAE在嵌入相似度上始终更高。改进幅度随扰动增强而增大，表明其能在提升表示稳定性的同时保留语义信息。代码已开源。
CDAE: Enhancing Perturbation Robustness in Pretrained Language Models with Contrastive Denoising
Pre-trained language models have significantly improved sentence representation learning, yet their embedding remain sensitive to semantic preserving textual perturbations such as synonym substitution, masking and word dropout. This work proposes a lightweight Contrastive Denoising Autoencoder (CDAE) that refines pre-trained BERT embedding by jointly optimizing contrastive and reconstruction objective to learn perturbation-invariant representation. We evaluate the proposed framework using multiple perturbation strategies with varying strengths and compare it against the original BERT embeddings and SimCSE. Experimental results show that CDAE consistently preserves higher embedding similarity under perturbations, with the improvements becoming more pronounced as framework effectively enhances representation stability while preserving semantic information, highlighting perturbation-invariant learning as a promising direction for improving sentence embeddings. The source code is publicly available at: https://github.com/ComputationIASBS/CDAE