Model 1017 trails Sol-xhigh in WeirdML benchmark comparison
The interesting signal is the trajectory comment: with heavy token budgets still unsaturated, frontier reasoning models may keep gaining from simply spending more compute per task. Practitioners tracking WeirdML results and reasoning-setting trade-offs can use this to decide which high-effort model configuration to benchmark next. The comparison between 1017 high and Grok 4.7 offers a concrete baseline for further testing.
A benchmark comparison on X shows model 1017 performing worse than Sol-xhigh and only on par with Grok 4.7. The comparison used the 1017 high reasoning setting. The poster notes the task consumed 50M tokens and that the model's trajectory does not look close to saturation, calling 50M tokens per task small for what the post terms 'Whale appetites'. The post also states Codex remains strong in the same evaluation.
1017 does much worse than Sol-xhigh, merely on par with Grok 4.7 otoh this is 1017 *high*, and this trajectory doesn't look like it's close to saturation. 50M tokens/task is nothing with Whale appetites. WeirdML remains strong. I don't think Codex matters a lot, it's strong there https://t.co/KY2TDsnIEh
- Artificial Analysis10-21 17:35原文
