AITOPAI热报
PapersSelected

EC²-Bench benchmarks edge-cloud collaboration for long-horizon agents

OpenBMB@OpenBMB
Post
View the post
Why we picked it

Cost and privacy trade-offs in agent execution do not move together — once task completion converges, strategy choice is mostly a privacy decision, and the best-scoring setup is also the most exposed. Teams deploying on-device agents can pick a collaboration mode based on edge model size: route execution to the cloud for small models, use cloud advisory for 9B-class ones. The comparison against Tsinghua-affiliated GLM-5.3 makes the findings directly applicable to Chinese open-source model stacks.

OpenBMB and Tsinghua researchers introduced EC²-Bench, a benchmark evaluating four edge-cloud collaboration strategies across 128 executable agent tasks. With Qwen3.5-9B on-device and GLM-5.3 in the cloud, the best collaboration reached 60.46 completion versus 39.12 edge-only, at US$8.73 versus US$56.42 for cloud-only. On Qwen3.7-Plus, strategy gaps in task completion shrank to 3.67 points, yet privacy exposure varied by 66.23 percentage points, with the top-performing strategy leaking the most. The optimal mode flipped with edge capability: 2B models scored 39.97 with per-step cloud routing versus 15.86 with self-delegation, while 9B models did best when the cloud only advised.

Image · OpenBMB
Original · OpenBMB

When should the cloud execute for the edge agent, and when should it just advise?

We present a systematic study of edge-cloud collaboration in long-horizon agent execution. Using EC²-Bench, we evaluate four collaboration strategies across 128 executable tasks, jointly measuring task completion, resource cost, and privacy exposure. Here are three key findings:

1️⃣ Small cloud budget, big gains.(cost-efficiency). With Qwen3.5-9B on-device and GLM-5.3 in the cloud, edge-only scores 39.12; the best collaboration reaches 60.46 — about 73% of cloud-only (82.70) — at only $8.73, roughly 15% of cloud-only's $56.42. 2️⃣ Performance converges, privacy doesn't. On Qwen3.7-Plus, best collaboration hits 76.84 vs 82.70 cloud-only; strategy gaps shrink to just 3.67 completion points, yet privacy exposure spans 66.23 percentage points — and the best-performing strategy leaks the most. 3️⃣ The mode flips with edge capability. Weak edges do best with an external router that lets the cloud execute (2B: Per-step Routing 39.97 vs self-delegation 15.86); stronger edges do best when the cloud only advises (9B Cloud Advisory 60.46). Further experiments highlight two abilities that matter for effective self-directed collaboration: knowing when to ask for help, and actually acting on the cloud's advice.

📄 Paper: https://t.co/ESc7nA1Aeu 🌐 Project: https://t.co/J1A3ZjtgCj 💻 Code: https://t.co/xjAuYnCDT1

#EdgeCloudCollaboration #AIAgent #LLM #OnDeviceAI #AgentBenchmark #EC2Bench #OpenBMB #Tsinghua

#EC2-Bench#OpenBMB#GLM-5.3#Qwen#edge-cloud collaboration#agents

Keep reading

Recent stories with the same tag or section
PapersBuildersSelected17:32
University of Chicago Paper Finds LLMs Misreport Their Own Cruxes

Sampling multiple answers and diffing them beats asking a model to explain itself — a cheap workflow change that applies to any high-stakes LLM use like research triage or forecasting. The 86x compute saving from the reranker makes it practical to run on everyday tasks. Practitioners building review pipelines or agent self-checks in Singapore and the wider ASEAN developer community can adopt this with any API-based model.

#LLM evaluation#reasoning models#Metaculus#University of Chicago
rohanpaul_ai@rohanpaul_aiSource
PapersBuildersSelected08:33
Meta and Carnegie Mellon RL-trained 8B Qwen3 beats 402B Maverick on tool picking

The result argues that tool selection can be moved off the flagship model entirely: a small RL-trained selector handles catalogs of 10,000+ tools that cannot fit in context, where retrievers alone rank only by relevance. Teams building agents with large tool catalogs can swap in a lightweight selector while keeping their main model unchanged. The performance gap over a prompted 402B model, 41.5% versus 30.6% exact picks, makes the cost saving concrete.

#Qwen3#Llama-4-Maverick#ToolBench#agents
Event
rohanpaul_ai@rohanpaul_ai2 sourcesSource