Papers

ToR-OT:用自回归 token 回归学习最优传输方案

Learning Optimal Transport Plans Via Autoregressive Token Regression

Why we picked it

把最优传输当成序列生成来解,预训练一次就能零样本处理新分布和异构成本,推理时不用跑优化器。虽然还没法替代经典求解器,但如果你研究摊销优化或神经求解器,这篇的稀疏解码思路值得借鉴。

论文提出 ToR-OT,一个基于 transformer 的模型,把最优传输(OT)问题重构为结构化序列生成任务,通过自回归 token 回归预测离散 OT 传输方案。作者在大量合成 OT 实例上预训练,单个模型即可 zero-shot 泛化到未见过的分布对和异构成本度量,无需重新训练。

Body

技术上的核心是利用精确离散 OT 解天然具有的稀疏性:解码器只预测非零条目,输出稀疏原始解,避免了稠密耦合矩阵的生成。推理阶段不再运行优化求解器,而是直接解码序列,实现摊销式推断;在低维离散问题上,ToR-OT 给出准确的近似方案,并优于按实例训练的神经 OT 基线。

作者明确指出 ToR-OT 尚不能替代经典求解器,经典 solvers 仍是该任务的金标准。论文的价值在于提供了一个有真值解、边际约束显式、稀疏性可刻画的测试床,用以研究自回归 transformer 能否内化此类约束优化问题的结构。

Original · arXiv cs.AI

Learning Optimal Transport Plans Via Autoregressive Token Regression

Optimal Transport (OT) provides a principled framework for finding the minimum cost coupling between probability distributions. Classical solvers remain the gold standard for this task. At the same time, OT is an ideal testbed for studying whether learning-based amortized models can internalize the structure of OT type problems. Here, ground-truth solutions are available and marginal constraints are explicitly defined. Also, exact discrete solutions have well-characterized sparsity. We describe ToR-OT, a transformer-based model that learns to predict discrete OT plans through autoregressive token regression. We reframe OT into a structured sequence generation task. ToR-OT decodes sparse transport plans as sequences instead of solving an optimization task at test time. We focus on three properties: (a) Zero-shot generalization: pretraining on a large synthetic corpus of OT instances allows a single model to generalize to unseen distribution pairs and heterogeneous cost metrics without retraining, (b) Sparse primal decoding: by exploiting the inherent sparsity in exact discrete OT solutions, the decoder predicts only nonzero entries (avoids dense coupling outputs), (c) Amortized inference: on low-dimensional discrete problems, ToR-OT gives accurate approximate plans and compares favorably with per-instance neural OT baselines. Overall, ToR-OT is not yet a replacement for classical solvers, but a step toward understanding how autoregressive transformers can learn structure of constrained optimization problems like OT.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Optimal Transport, ToR-OT, transformer.

Source attribution: curated from arXiv cs.AI (@Ivan Jaen Marquez, Sotirios Panagiotis Chytas, Vikas Singh).