Primary sourceBindu Reddy
View sourceEvent ·Multi-source
开发者上线首日将 Gemini Neon 与 Claude Opus 5.5 做盲测 A/B 对比
开发者 Bindu Reddy 宣布在产品上线第一天就把 Gemini Neon 和 Claude Opus 5.5 放进生产环境做盲测 A/B 实验,用真实用户流量来判断两个模型的实际表现。 她同时给出了一个明确的决策规则:如果 Claude Opus 5.5 赢下这轮对比,就买入 Anthropic 的股票;如果 Gemini Neon 胜出,则放弃这笔投资。也就是说,模型评测结果直接决定她的仓位。 这个做法的参考价值在于把模型选择从跑分和感觉,变成用真实生产流量验证的量化对比,其他开发者在选型时可以借鉴同样的盲测方法。
Current read
这条推文有意思的地方是作者直接在生产环境做盲测,还把结果和买不买 Anthropic 股票绑在一起。选模型别只看跑分,学她用真实流量 A/B 一下更靠谱。
11 sources31° AI heatLast updated 2026/11/19 04:01:48
Evidence
11 sourcesRelated source 1The Rundown AI
View sourceRelated source 2TestingCatalog
View sourceRelated source 3Artificial Analysis
View sourceRelated source 4Tech in Asia
View sourceRelated source 5Latent Space (swyx)
View sourceRelated source 6IT之家
View sourceRelated source 7Gergely Orosz
View sourceRelated source 8arXiv: Anthropic
View sourceRelated source 9OpenAI
View sourceRelated source 10Yuchen Jin
View sourceConflict check
Dedup data only shows that these sources discuss the same event, not that they agree. There is no structured supporting or refuting evidence yet, so no conflict is inferred automatically.