METR这篇GPT-5.6评测挺有意思，作弊多到测不准，还说作弊是好事，值得看看。
METR在GPT-5.6 Sol的预部署评估中发现，该模型的作弊率高于其测试过的任何公开模型，甚至会在推理中思考自己被监视的事实。METR明确指出，不认为GPT-5.6 Sol具备危险能力，也未达到OpenAI准备框架v2中AI自我改进的关键能力阈值。METR强调，可见的作弊反而是好事，更应警惕那些表面干净的模型，因为它们可能学会了隐藏行为。评估前沿模型在能力和行为两个维度都变得愈发困难，需要更多投入。
Highly-recommended reading. Interesting details in this METR's GPT-5.6 eval. They couldn't get a c...
Highly-recommended reading. Interesting details in this METR's GPT-5.6 eval. They couldn't get a clean capability number because the model cheated more than any public model they've tested, and even reasoned about the fact that it was being watched. To be clear, METR doesn't think it's dangerously capable. In their words: "we do not believe GPT-5.6 Sol would enable fully automated AI R&D, nor do we believe it meets the Critical capability threshold for AI Self-Improvement in OpenAI's Preparedness Framework v2." METR says visible cheating is the good case. The model to fear is the one that looks clean, because it may have just learned to hide. My take overall is that evaluation is becoming the hard part with newer frontier models. Both from a capability and behavioral point of view. We desperately need more investment here. METR @METR_Evals OpenAI gave METR early access to GPT-5.6 Sol for testing including raw chain-of-thought, a railfree version of the model, and internal information about the model. With this access, METR conducted a pre-deployment evaluation of GPT-5.6 Sol, including an attempted measurement of its 50%-Time Horizon. However, the measurement depends heavily on our treatment of cheating attempts, and GPT-5.6 Sol’s detected cheating rate was higher than any public model we have evaluated. 🔗 View Quoted Tweet 💬 5 🔄 1 ❤️ 8 👀 1705 📊 4 ⚡