ViTAMINS用合成难例提升视觉Transformer性能，资源效率还更高，ViT-B就能打败V-JEPA的ViT-L。
ViTAMINS方法将合成难例整合到无监督视觉Transformer预训练中，提升表示质量。在ImageNet和迁移学习、图像检索、拷贝检测、图像视频分割任务上全面基准测试。ViTAMINS的难例带来涌现特性，学习到的表示包含图像语义内容信息，作为分类器表现优异，比基线提升最高11.3%。ViTAMINS通过简单修改现有对比框架实现，资源效率更高，ViT-B版本超越V-JEPA的ViT-L版本。
ViTAMINS: An Empirical Study of Training Self-Supervised Vision Transformers with Synthetic Hard Negatives
We introduce ViTAMINS, a method that integrates synthetic hard negatives into unsupervised vision transformer pretraining to improve representation quality. Our approach is thoroughly benchmarked on ImageNet and transfer learning, image retrieval, copy detection, and image, video segmentation tasks. Notably, our proposed negatives give rise to emergent properties, where learned representations contain explicit information about the semantic content of an image and serve as excellent classifiers (up to +11.3% over baselines). ViTAMINS achieves these benefits through simple modifications to existing contrastive frameworks and outperforms competing methods while being more resource efficient, e.g., our ViT-B surpasses V-JEPA with ViT-L. Our findings motivate reconsidering contrastive learning as a simpler yet powerful alternative to dominant generative and self-distillation approaches.