做单细胞转录组分析的团队终于有了一个能同时提升性能和可解释性的Transformer方案——scTransformer把基因调控先验直接嵌入注意力机制，比黑盒模型更可信，建议做生物信息学基础模型的研究者点开看看。
研究人员提出scTransformer，这是首个将已知基因调控关系作为先验知识融入Transformer注意力机制的方法。通过约束信息流遵循已知调控结构，模型学习到的细胞表示更具生物学意义。在疾病相关的单核RNA-seq数据集上，scTransformer在细胞类型分类任务中提升了准确率，增强了嵌入空间中细胞类型的分离度，并产生了与已知调控程序一致的注意力模式。该方法在不牺牲性能的前提下增强了模型可解释性，为构建生物学基础的单细胞组学基础模型迈出了原则性的一步。
Integrating gene regulatory priors into Transformer attention with scTransformer for interpretable scRNA-seq analysis
Motivation: Transformer-based models are increasingly applied to large-scale single-cell transcriptomics, showing strong performance through self-supervised learning on millions of cells. However, most existing approaches treat genes as independent features, and largely ignore prior biological knowledge, which limits interpretability and robustness. In this paper, we explore whether explicitly incorporating gene regulatory information can improve both model performance and biological insight. Results: We present scTransformer, the first Transformer-based approach that builds a priori knowledge of biological mechanisms into the model's attention patterns. By constraining information flow according to known regulatory structures, the model learns representations that are more biologically meaningful. We evaluate scTransformer on a disease-relevant single-nucleus RNA-seq dataset using supervised cell-type classification. Compared to standard Transformers, our approach improves classification accuracy, enhances separation of cell types in embedding space, and produces attention patterns consistent with known regulatory programs. Overall, our results demonstrate that embedding biological structure into Transformer models can enhance interpretability without sacrificing performance, offering a principled step toward biologically grounded foundation models for single-cell omics.