做智能体系统部署和运维的团队会直接受益——这篇论文给出了一个实用的监控框架，帮你区分结构缺陷和任务错误，减少无效告警。建议点开看看，尤其是处理文档密集型工作流的团队。
本文提出了一种针对生产环境中智能体系统的监控与分类方法，这些系统通常存在结构缺陷而非任务级错误。该方法从质量、适用性和效率三个维度，在运行内、跨运行和结构三个范围进行监控，利用变异系数作为特征信号。通过合成测试床（220次运行，120个文档包）验证，发现结构缺陷会掩盖任务级错误信号，而确定性分类可将97%的结果自动跟踪，仅2%需要人工调查。论文建议在集成缺陷解决后，监控应从结构表征过渡到错误检测再到可靠性跟踪。
Monitoring Agentic Systems Before They're Reliable
Agentic systems entering production typically operate as partially integrated assemblies where structural defects, not task-level errors, dominate the failure landscape. At this maturity level, task-level error detection may be infeasible: structural failure modes mask the signal that task-level monitors are designed to detect.We present a monitoring and triage methodology that decomposes agentic system evaluation into three dimensions (quality, suitability, efficiency) at three monitoring scopes (within-run, cross-run, structural), using variance as a characterization signal. Findings are routed through severity classification adapted from FMEA, concentrating human attention on the subset that warrants investigation. We evaluate on a synthetic testbed of 220 runs across 120 document bundles with controlled error injection.Three results emerge. Monitor scope determines failure type: within-run monitors surface deterministic stage defects (CV = 0.02), cross-run monitors surface stochastic integration consequences (CV = 1.25, 24% at L2), and a structural monitor identifies an integration gap with perfect consistency (CV = 0.00). Injected task-level errors are indistinguishable from clean baselines, confirming structural defects mask task-level signal. Deterministic triage routes 97% of findings to automated tracking, leaving the 2% reflecting variable behavior for human investigation.We propose, on Stage 1 evidence, a maturity-staging model in which monitoring transitions from structural characterization to error detection to reliability tracking as integration defects resolve. The taxonomy, CV-based scope characterization, and severity model transfer architecturally to document-driven, multi-stage agentic workflows in regulated industries; specific calibrations are domain-specific. Deploy monitoring early: the first thing it finds is the most important thing to fix.