做算子学习或 PDE 求解的团队，终于有办法让 Transformer 不再把连续场当离散 token 处理了——Functional Attention 用函数对应替代 token 注意力，既提升泛化又保持分辨率不变，值得一试。
本文提出 Functional Attention，一种将 Transformer 注意力机制重新解释为自适应基函数之间函数对应关系的方法。受几何函数映射启发，该方法用结构化线性算子替代 softmax 亲和度，从而获得紧凑、可泛化、分辨率不变的表示，显式捕捉全局依赖。实验表明，在求解 PDE、3D 分割和回归等算子学习任务中，Functional Attention 达到最先进性能，且对不同离散化方式保持鲁棒。项目代码已开源。
Functional Attention: From Pairwise Affinities to Functional Correspondences
Learning mappings between infinite-dimensional function spaces, or operator learning, is essential for many machine learning applications. Although transformer-based operators are popular, they often rely on token-wise attention. These methods treat continuous fields as discrete tokens and usually ignore the global functional structure. We introduce \emph{Functional Attention}, which reinterprets attention as a functional correspondence between adaptive bases. Inspired by geometric functional maps, our method replaces softmax affinities with structured linear operators. This yields a compact, generalizable, resolution-invariant representation that explicitly captures global dependencies. Experiments demonstrate that \emph{Functional Attention} can match state-of-the-art performance in many operator learning tasks, including solving PDEs, 3D segmentation, and regression, while remaining robust to varying discretizations. Project page is available at https://github.com/xjffff/FUNCATTN.