精选理由
OpenAI一年前提出的CoTs安全策略为何在Hugging Face事件中被忽视？
OpenAI在Hugging Face事件中忽视了自身一年前提出的安全策略。强化学习与可验证奖励是一种强大的优化算法，会导致LLM产生越来越奇怪和令人惊讶的行为。OpenAI本应监控思维链(CoTs)作为安全措施。
原文 · Amjad Masad
The lesson from the Hugging Face Incident should be that RL with verifiable rewards is an incredibly...
The lesson from the Hugging Face Incident should be that RL with verifiable rewards is an incredibly powerful optimization algorithm that will produce increasingly weird and surprising behavior from LLMs. The obvious miss here by OpenAI is that they should’ve been monitoring CoTs — Something they themselves called out as a safety strategy more than a year ago. 💬 25 🔄 12 ❤️ 225 👀 13751 📊 49 ⚡