这篇论文用Ising模型改造注意力，在逆合成预测上吊打标准方法，做科学AI的必看。
本文提出Variational-Ising-Attention (VIA)，在softmax归一化中引入Ising模型相互作用，通过变分平均场推断学习可调耦合参数。在逆合成反应中心预测任务上，VIA相比标准softmax注意力一致且显著提升性能。实验覆盖多种模型变体和机制分析，验证了为科学问题定制注意力的有效性。
Variational-Ising-Attention (VIA):TailoredAttentionMattersfor Science
Attention enables context modeling via query-key scoring with softmax normalization. Driven by industrial long-context demands, mainstream research has converged toward sparsity and efficiency--yet softmax's independence assumption persists. For scientific tasks unburdened by long-token constraints, however, richer structured coupling may often be essential, making tailored attention both viable and more appropriate. To this end, we propose Variational-Ising-Attention (VIA), which augments softmax normalization with an interacting Ising model; attention patterns emerge from learnable pairwise couplings via variational mean-field inference, redefining attention from a ranking over isolated items to a collective state over interacting entities. We instantiate VIA on retrosynthesis reaction center prediction, a task inherently governed by cooperative bond-breaking constraints. Comprehensive experiments across model variants, coupled with mechanistic analyses, demonstrate that VIA consistently and substantially outperforms standard softmax attention. More broadly, our findings suggest that for scientific problems, the optimal solution is not general-purpose efficiency, but appropriately tailored attention aligned with intrinsic domain structure. This work provides a theoretically grounded and empirically validated instantiation of this paradigm.