这篇论文用对比学习做EEG预训练，比传统掩码重构效果好，架构设计很实在，搞脑机接口的可以看看。
该论文提出CoCoT-EEG，一种对比预训练的多尺度卷积Transformer模型，用于脑电图解码。在多个异构电极配置的基准任务上，CoCoT匹配或超越了基于掩码重构预训练的最先进模型。从零开始训练的CoCoT也优于先前单任务解码模型，展现出数据效率。通过系统消融实验，作者验证了对比学习用于构建EEG基础模型的可行性，并指出关键架构设计考量。
CoCoT-EEG: Contrastive-Pretrained Multiscale Convolutional Transformer for EEG Decoding
Self-supervised pretrained foundation models (FM) have shown early promise for non-invasive electroencephalogram (EEG) decoding applications. Many recent large-scale models converged on the approach of tokenizing raw EEG followed by masked reconstruction pretraining. However, this recipe has been shown to be suboptimal for data, like EEG, with high noise amplitude and information confined to limited dimensions such as narrow frequency bands. Building on this insight, we develop a novel contrastive-pretrained EEG model with multiscale temporal convolution input layers and Transformer encoder blocks (CoCoT). CoCoT matches or beats state-of-the-art reconstruction-pretrained EEG models on extensive benchmark decoding tasks with heterogeneous electrode configurations. Furthermore, CoCoT trained from scratch outperforms previous single-task decoding models and even rivals pretrained models, showcasing the architecture's flexibility and data efficiency. Through systematic ablations, including model architecture and pretraining objective, we demonstrate the viability of contrastive learning for building EEG FMs while suggesting key architectural design considerations, prompting further investigations in alternative large-scale pretraining strategies.