这篇论文把“模型为什么有性能天花板”从信息论和交互角度讲得很清楚，做LLM智能体的可以看看它怎么把系统当随机过程分析。
这篇论文从信息论视角分析机器学习决策系统的内在性能上限。作者用Fano型下界刻画分类的最小误差，用Cramér-Rao不等式给出参数估计的精度极限。研究指出，独立假设、遍历性和分布稳定性等隐含条件会显著影响推断结论的有效性。论文还基于马尔可夫随机场和势函数表示来建模数据中的依赖机制，并把LLM集成智能体架构视为反馈驱动的随机过程。最终强调预测能力扩展的前提是建立充分的数据模型，算法学习必须服从信息论极限。
On the Structural Limits of Machine Learning Decision Systems: An Information-Theoretic, Interaction-Based, and Stochastic-Dynamical Perspective
Machine learning procedures are commonly evaluated in terms of predictive accuracy and computational efficiency. However, their achievable performance is fundamentally constrained by structural properties of the underlying data-generating process, which are formalized in terms of informational bounds. In this work we examine intrinsic limits of data-driven decision systems from an information-theoretic and interaction-based perspective. We analyze minimal achievable error in classification through Fano-type bounds and precision limits in parametric estimation via the Cramér-Rao inequality, emphasizing that such limits depend on the underlying model rather than on algorithmic sophistication alone. We further discuss how implicit assumptions, such as independence, ergodicity, and distributional stability, affect the validity of inferential procedures. Building on interaction-based modeling principles, we review typical frameworks such as Markov Random Fields and potential based representations for encoding dependence mechanisms. We also describe decision systems, including LLM-integrated agent architectures, as feedback-driven stochastic processes where state-dependent dynamics may induce emergent macroscopic behavior. This perspective highlights the importance of having adequate models for the data as a prerequi- site for expanding predictive capability, and situates algorithmic learning within the informational limits imposed by the models.