这篇论文提出了BrainJanus，一个能双向翻译脑信号与图像、文本的统一模型，在零样本和生物可解释性上突破传统方法。
BrainJanus是首个将脑、视觉和语言整合到单一框架的统一脑模型。它引入Unified Brain Tokenizer将连续神经活动量化为离散Token，并与视觉和语言表征对齐到共享的Omni空间。基于All-in-One自回归架构，该模型通过下一个Token预测实现图像到脑、文本到脑的编码以及脑到图像、脑到文本的解码。在多项基准测试中，BrainJanus取得优越性能，并展现出零样本泛化能力和可解释的生物拓扑结构。代码已在GitHub开源。
BrainJanus: A Unified Model for Understanding and Generation across Brain, Vision, and Language
Modeling the bidirectional correspondence between external sensory stimuli and internal neural activity has emerged as a critical frontier in neuroscience. However, existing approaches predominantly treat brain encoding and decoding as isolated tasks, relying heavily on unimodal alignment and external priors while overlooking the brain's intrinsic nature as a multimodal integration system. To address these limitations, we propose BrainJanus, the first unified brain model that integrates brain, vision, and language within a single framework. Specifically, we introduce a Unified Brain Tokenizer to quantize continuous neural dynamics into discrete tokens aligned with visual and linguistic representations in a shared Omni space. Building on this, we utilize an All-in-One autoregressive architecture that leverages next-token prediction to enable seamless any-to-any generation, which encompasses image-to-brain and text-to-brain encoding, and brain-to-image and brain-to-text decoding. Extensive experiments demonstrate that BrainJanus achieves superior performance across diverse benchmarks. Furthermore, our framework exhibits zero-shot generalization and preserves interpretable biological topography, highlighting its potential as a general-purpose brain modeling paradigm. The code is available at \href{https://github.com/HaitaoWuTJU/BrainJanus}{GitHub}.