Latest

Every published item, in order · 568 items
Oct 26
Oct 25
Oct 24
Oct 23
ModelsBuildersSelected08:21
Artificial Analysis Coding Agent Index v2.0: Claude Opus 5.5 Leads

The v2.0 changes make rankings harder to game: private test sets, anti-solution scanning and multi-vote LLM judging push mean scores down 1.7 points across 21 shared configurations. Teams choosing a coding agent should note that Opus 5.5's edge is concentrated on FrontierCode (56.0% versus 52.6% for DeepSeek V4.1 Flash), while 15 of 24 configurations cluster between 43% and 49%, so harness choice can matter as much as model choice.

官方账号Artificial Analysis@ArtificialAnlysSource
ModelsUS vs ChinaMulti-sourceSelected01:57
CritPt v2 Benchmark Launches with Agentic Code Execution

The 54-point gap between GPT-5 and GPT-6 Astra in about a year shows how quickly frontier models are absorbing research-level physics when given code execution. Tracking these scores before the late October Intelligence Index update helps researchers and benchmark watchers pick models for scientific workflows. Compare against Claude Opus 5.5 at 78% when choosing a frontier model for agentic computation tasks.

Event
官方账号Artificial Analysis@ArtificialAnlys11 sourcesSource
Oct 22
PapersBuildersMulti-source11:01
Chinese content六款前沿模型实测 LIBERO-10 机器人操作基准,GPT-6 Astra 以 44% 成功率居首

研究者让 GPT-6、Claude Opus 5.5 这些模型直接操控真实机器人跑 LIBERO-10,1,800 次测试排出了名次,最强也只有 44% 成功率,想了解大模型动手能力差在哪的可以看。

Event
arXiv cs.AI@May Espinola et al. (3)10 sourcesSource
Oct 21
IndustryUS vs ChinaMulti-sourceSelected00:31
Intelligence Index thresholds now far cheaper per task, Artificial Analysis finds

The price of intelligence is collapsing at every fixed quality level while top-end capability prices keep climbing, so buyers face a widening spread between commodity and frontier performance. Teams benchmarking models should match their task's required score to the cheapest threshold model instead of defaulting to the highest scorer. The data also names concrete price leaders at each level, such as Claude Haiku 5.5 (xhigh) at US$0.12/task for a score of 41.

Event
官方账号Artificial Analysis@ArtificialAnlys11 sourcesSource
Oct 20
ModelsBuildersSelected23:05
Kaggle launches Human Behavior Bench for predicting human decisions

A 94.1% score above the human noise ceiling suggests behavioural-simulation systems like Engram 1.0 can outperform frontier LLMs at modelling subgroups, not just answer questions. Product and research teams building personalisation, market research or user modelling tools now have a public baseline to test against. The 601 private cases make leaderboard gaming harder, so results should hold up better than fully public benchmarks.

Kaggle@kaggleSource