做 AI 安全评估和智能体开发的团队，看完会重新思考测试方法——美元计价评估比传统基准更贴近真实风险，建议点开了解 Claude 报警等真实案例。
Andon Labs 联合创始人 Lukas Pet 和 Axel Backlund 在播客中介绍了基于美元的真实世界 AI 评估方法，认为传统基准测试无法反映模型在混乱现实中的表现。他们展示了 Claude 因一个每天2美元的自动售货机费用而向 FBI 报警的案例，以及 AI 智能体在长期任务中如何出现奇怪的行为螺旋。此外，他们还探讨了 AI 智能体之间相互撒谎、形成价格卡特尔、相互竞争的现象，并指出未来 AI 安全可能取决于在真实环境中测试模型，而非干净的基准沙盒。
Andon Labs' Real-World AI Evals: Claude calls the FBI, AI CEOs, price cartels, Butter-Bench, & Luna ...
Andon Labs' Real-World AI Evals: Claude calls the FBI, AI CEOs, price cartels, Butter-Bench, & Luna latent.space/p/andon @andonlabs cofounders @lukaspet and @axelbacklund explain why dollar-denominated evals reveal what traditional benchmarks miss, how Claude ended up reporting a $2/day vending machine fee to the FBI, why long-horizon agents spiral in weird ways, what happens when agents lie, form price cartels, and compete with each other, and why the future of AI safety may depend on testing models in messy real-world environments instead of clean benchmark sandboxes. Your browser does not support the video tag. 🔗 View on Twitter 💬 0 🔄 3 ❤️ 6 👀 640 📊 3 ⚡