PapersSelected

TokenWave.AI 发布 SWE-CoWork:测试智能体协作交接能力的新基准

Why we picked it

做智能体开发的朋友可以看看这个新基准。SWE-CoWork 模拟了一个有任务板、文档和 26 个虚拟同事的工作场景,让智能体接手别人干到一半的活。结果挺扎眼:101 个任务最好的模型只完成 8 个,连 Claude Opus 5.5 都在拿到需求后丢要求。想找评测协作型智能体的方法,这套框架是现成的。

TokenWave.AI 发布 SWE-CoWork 基准,专门评估编码智能体在模拟团队环境中的工作交接能力。基准包含 101 个真实开源仓库的 101 项任务,覆盖 11 种编程语言、12 个应用领域和 17 种职场工具。

Body

每项任务从一个简短的交接开始,智能体需要在模拟职场中搜索任务板和版本化文档,向 26 个模拟同事询问缺失细节,并在回复改变需求时修改代码,最终提交通过隐藏测试的补丁。测试结果很严峻:表现最好的模型只完整完成 101 项任务中的 8 项,Claude Opus 5.5 有超过一半的失败检查发生在已获得需求之后,说明开放性问题和已确认需求在交接过程中都会丢失。

这个基准直指当前智能体开发中被忽视的环节:让智能体像数字员工一样接手同事未完成的工作。论文、代码和评测框架已在 arXiv(2611.20218)、GitHub 和 Hugging Face 上公开,开发者可以直接复现或扩展这套评测。

Original · elvis

Handoff capabilities are something I am looking at closely with agents. They are still hard to code for agents. SWE-CoWork measures this by putting an agent in a simulated team with tickets, docs, and teammates it can ask. It then has to deliver a patch that passes hidden tests across 101 real repos. The best model fully finished 8 of 101 tasks. For Claude Opus 5.5, more than half of the failed checks came after the agent already had the requirement. Great work from @TokenWaveai . TokenWave.AI @TokenWaveai Can a coding agent take over a teammate’s work in progress? Introducing SWE-CoWork, a benchmark for coding agents working as digital employees on long-horizon software engineering tasks. Each assignment starts with a short handoff in a simulated workplace modeled on real-world engineering workflows. Agents search task boards and versioned docs, ask teammates for missing details, and revise their code when a reply changes what needs to be built. Open questions and confirmed requirements can both be lost before the agent hands the work back. SWE-CoWork lets us trace them through the conversation, the patch, and the tests. 101 assignments across 101 real open-source repositories. 26 simulated teammates per workplace. 11 programming languages. 12 application domains. 17 workplace tools. 🌐 Website: tokenwave-ai.github.io/swe-cowork-pag… E 📄 Paper arxiv.org/abs/2611.20218 Gy 💻 Cod github.com/TokenWave-AI/s… Wir 🤗 Hugging Face Daily Pape huggingface.co/papers/2611.20… 4pKR (If you find it useful, we’d appreciate an upvote!)

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include SWE-CoWork, Claude Opus 5.5, TokenWave.AI, agents.

Source attribution: curated from elvis (@omarsar0). Cross-source corroboration: 2 independent sources have covered this development (including arXiv cs.AI), confirming industry-wide signal strength.