Agent Initiative Benchmark by Deepnote Ranks GPT-6 Astra First
The benchmark shifts evaluation focus from task completion to initiative judgement — knowing when not to act — a gap standard QA suites do not measure. Teams evaluating proactive agents for internal workspaces get a comparable cost-per-task metric alongside scores, which exposes Claude Fable 5.2's steep price gap versus the GPT-6 family. Builders choosing models for Slack integrations and agent orchestration can use the loop design as a template for their own internal evals.
Deepnote (@deepnotehq), an agent workspace company, released the Agent Initiative Benchmark, a test that drops proactive agents into a simulated company Slack and scores whether they know when to intervene, what to say, and when to stay quiet.
Every model runs through the same loop: a thread starts in a simulated Slack workspace, the agent decides whether to act at all, chooses which of 11 services to check and investigates, then either posts and tags the right people or stays silent. Two LLM evaluators score each run 0 to 10 against a reference answer.
GPT-6 Astra led with 8.97 out of 10, and GPT-6.1 Sol followed at 8.65 for US$0.11 per task, against US$1.87 per task for third-placed Claude Fable 5.2. The benchmark targets a capability older QA suites ignore: initiative judgement in workplace collaboration, where acting wrongly is often worse than not acting at all. For teams deploying proactive agents in Slack-like environments, cost per intervention and restraint behaviour may matter as much as raw capability scores.
This is a beautiful concept for a benchmark.
Deepnote (@deepnotehq), an agent workspace company, released the Agent Initiative Benchmark, a test that drops proactive agents into a company Slack and scores whether they know when to jump in, what to say, and when to stay quiet.
GPT-6 Astra led with 8.97 out of 10, and GPT-6.1 Sol followed at 8.65 for $0.11 per task, against $1.87 per task for third-placed Claude Fable 5.2.
It runs every model through the same loop:
> A thread starts in a simulated company’s Slack > The agent decides whether to act at all > It picks which of 11 services to check and investigates > It posts and tags the right people, or stays silent > Two LLM evaluators score the run 0 to 10 against a reference answer
This report is categorized under AITOP's "Products" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include Deepnote, GPT-6, Claude, agents.
Source attribution: curated from rohanpaul_ai (@rohanpaul_ai). Cross-source corroboration: 11 independent sources have covered this development (including lmarena.ai, Harrison Chase, TestingCatalog), confirming industry-wide signal strength.
