TxFM解决了转录组基础模型性能不佳的痛点，做药物发现和生物信息学的团队可以试试这个轻量级方案，效果优于大模型。
RNA测序数据在药物发现中有广泛应用，但现有转录组基础模型因技术噪声和批次效应表现不佳，甚至不如线性基线。研究者提出新自监督模型TxFM，采用掩码自编码方法处理多样RNA-seq计数数据，并通过消融实验确定关键架构配置。他们构建了公开训练语料DiverseRNA-1.4M，训练出的TxFM在基因表示上优于规模大100倍以上的图谱级模型。结果表明，结合精心设计的模型架构和训练数据，归纳式自监督学习是转录组表示学习的可行方法。
Effective Biological Representation Learning by Masking Gene Expression
RNA sequencing produces rich and diverse datasets of gene expression, offering compelling insights into cellular state and function that have many applications in drug discovery. Modeling such data is challenging due to inherent technical noise and experimental batch effects, as evidenced by many existing transcriptomic foundation models (FMs) underperforming relative to linear baselines. Such results raise the question of whether deep representation learning provides a distinct advantage over the direct use of raw transcript counts. Our work explores this by developing a new self-supervised model, TxFM, with a focus on inductive representation learning evaluations. TxFM employs a masked autoencoding approach tailored to diverse RNA-seq count data, and our ablation study empirically identifies crucial architecture configurations required for strong transfer performance. Additionally, we curate a public training corpus, DiverseRNA-1.4M, and find that TxFM trained on this curated dataset yields high-fidelity gene representations that outperform FMs trained on atlas-scale corpora over 100x larger. Overall, our results indicate that inductive self-supervised learning is a viable modeling approach for transcriptomics representation, provided a careful synthesis of model architecture and training data curation.