TokenWave.AI 发布 SWE-CoWork:测试智能体协作交接能力的新基准
做智能体开发的朋友可以看看这个新基准。SWE-CoWork 模拟了一个有任务板、文档和 26 个虚拟同事的工作场景,让智能体接手别人干到一半的活。结果挺扎眼:101 个任务最好的模型只完成 8 个,连 Claude Opus 5.5 都在拿到需求后丢要求。想找评测协作型智能体的方法,这套框架是现成的。
TokenWave.AI 发布 SWE-CoWork 基准,专门评估编码智能体在模拟团队环境中的工作交接能力。基准包含 101 个真实开源仓库的 101 项任务,覆盖 11 种编程语言、12 个应用领域和 17 种职场工具。
每项任务从一个简短的交接开始,智能体需要在模拟职场中搜索任务板和版本化文档,向 26 个模拟同事询问缺失细节,并在回复改变需求时修改代码,最终提交通过隐藏测试的补丁。测试结果很严峻:表现最好的模型只完整完成 101 项任务中的 8 项,Claude Opus 5.5 有超过一半的失败检查发生在已获得需求之后,说明开放性问题和已确认需求在交接过程中都会丢失。
这个基准直指当前智能体开发中被忽视的环节:让智能体像数字员工一样接手同事未完成的工作。论文、代码和评测框架已在 arXiv(2611.20218)、GitHub 和 Hugging Face 上公开,开发者可以直接复现或扩展这套评测。
Handoff capabilities are something I am looking at closely with agents. They are still hard to code for agents. SWE-CoWork measures this by putting an agent in a simulated team with tickets, docs, and teammates it can ask. It then has to deliver a patch that passes hidden tests across 101 real repos. The best model fully finished 8 of 101 tasks. For Claude Opus 5.5, more than half of the failed checks came after the agent already had the requirement. Great work from @TokenWaveai . TokenWave.AI @TokenWaveai Can a coding agent take over a teammate’s work in progress? Introducing SWE-CoWork, a benchmark for coding agents working as digital employees on long-horizon software engineering tasks. Each assignment starts with a short handoff in a simulated workplace modeled on real-world engineering workflows. Agents search task boards and versioned docs, ask teammates for missing details, and revise their code when a reply changes what needs to be built. Open questions and confirmed requirements can both be lost before the agent hands the work back. SWE-CoWork lets us trace them through the conversation, the patch, and the tests. 101 assignments across 101 real open-source repositories. 26 simulated teammates per workplace. 11 programming languages. 12 application domains. 17 workplace tools. 🌐 Website: tokenwave-ai.github.io/swe-cowork-pag… E 📄 Paper arxiv.org/abs/2611.20218 Gy 💻 Cod github.com/TokenWave-AI/s… Wir 🤗 Hugging Face Daily Pape huggingface.co/papers/2611.20… 4pKR (If you find it useful, we’d appreciate an upvote!)
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include SWE-CoWork, Claude Opus 5.5, TokenWave.AI, agents.
Source attribution: curated from elvis (@omarsar0). Cross-source corroboration: 2 independent sources have covered this development (including arXiv cs.AI), confirming industry-wide signal strength.
- arXiv cs.AI11-15 19:39Source