Sparse MoE Routing Geometry Study
The findings help optimize MoE model efficiency by revealing reusable routing processes across layers, potentially reducing computational costs.
The findings help optimize MoE model efficiency by revealing reusable routing processes across layers, potentially reducing computational costs.
The study reveals how AI models process moral concepts, relevant for developers building ethical AI systems in ASEAN.
这篇论文用几何视角解释LLM评判偏见,发现偏见隐藏状态的低维子空间,还能预测模型在未知基准上的失败,挺有意思。
做分布对齐或几何数据处理的团队,CDOT 解决了传统 GW 非凸优化不稳定的痛点,可以直接用这个凸框架提升效果,建议点开看理论证明和实验对比。
对理解大模型内部机制的研究者来说,这篇论文揭示了计数任务背后的几何结构,值得深入阅读。