Event ·Multi-source

Ben Hylak 谈用模拟做 AI 智能体评测

Hamel Husain 在 X 上发布了对 Ben Hylak 的访谈，主题是 AI Evals 课程中如何用模拟做评测。Ben 分享了在多家公司落地模拟评测的经验，指出输入输出测试会漏掉智能体的失败。他提到"模拟感知"现象：模型能察觉自己身处模拟环境并改变行为。访谈还覆盖回放生产 trace、重建智能体周边的数据与工具、跨运行检查答案与成本等环节。

Current read

Ben Hylak 讲怎么用模拟测智能体，还聊到模型会察觉自己在被模拟然后改行为，做 eval 的可以看看。

3 sources38° AI heatLast updated 2026/10/10 22:56:57

Evidence

3 sources

Primary sourceHamel Husain

2026/10/10 22:56:57View source

Related source 1shao__meng

2026/10/12 00:47:12View source

Related source 2Philipp Schmid

2026/10/12 16:07:09View source

Conflict check

Dedup data only shows that these sources discuss the same event, not that they agree. There is no structured supporting or refuting evidence yet, so no conflict is inferred automatically.

Open the story