这篇论文通过可视化展示了ViT训练过程中的特征演变和迁移，为理解ViT的学习和演变提供了新的视角。
本文通过可视化训练过程，使用稀疏自编码器提取候选稀疏特征，研究视觉Transformer（ViT）中特征级别的动态变化。实验表明，特征迁移主要发生在训练早期，且随着特征组织稳定而减少。深层网络比浅层网络更早和更稳定地稳定。
Feature Evolution and Migration during Vision Transformer Training
We present a novel view on feature evolution in Vision Transformers (ViTs) by visualizing the training process over two dimensions -- network depth (layer) and training time (epochs). We employ Sparse Autoencoders (SAEs) to extract candidate sparse features from CLS-token representations and compare their activation profiles across epoch--layer pairs. This allows us to study feature-level dynamics that are not directly visible from representation-level similarity measures. Furthermore, we demonstrate how this framework of feature evolution allows us to describe feature migration, the change in the layer where a feature is most detectable during training. Our experiments show that migration is concentrated early in training, occurs more often toward earlier layers than toward deeper layers, and declines as feature organization stabilizes. We further find that deeper layers stabilize earlier and more strongly than shallow layers. The results show that our approach can be employed as a tool for understanding how ViTs learn and evolve.