精选理由
北大和StepFun搞了个TensorCast，把大模型首Token延迟砍掉九成多，高并发多轮对话场景下很猛。
北京大学、StepFun与北京邮电大学提出TensorCast，一种面向大模型基础设施的统一可编程张量生命周期管理抽象。在高并发多轮智能体场景中，TensorCast将中位首Token延迟最高降低93.2%。模型实例启动速度最高提升228.6倍。该方案由三所机构联合提出。
原文 · pandaily
Peking University and StepFun Unveil TensorCast: A Programmable Tensor Management Layer That Cuts LLM Time-to-First-Token by Up to 93.2%
Peking University, StepFun, and Beijing University of Posts and Telecommunications propose TensorCast, a unified programmable tensor lifecycle management abstraction for large model infrastructure. In high-concurrency multi-turn agent scenarios, median time-to-first-token drops up to 93.2%, and model instance startup is up to 228.6x faster.