Latest

Every published item, in order · 470 items
Sep 29
Sep 28
ModelsBuildersMulti-sourceSelected72°00:51
OpenScience agent outscores Codex and Claude Code on science benchmarks

An Apache 2.0 agent beating proprietary coding agents on science benchmarks changes the calculus for labs that cannot ship data to closed platforms. Teams running research automation can self-host it and plug in whichever model they already pay for. The comparison set — Codex and Claude Code — makes the leaderboard claims directly checkable against tools practitioners use daily.

Event
rohanpaul_ai@rohanpaul_ai11 sourcesSource
Sep 27
How-toBuildersMulti-sourceSelected72°05:48
Jev Router Experiment Shows Benchmark Limits in Agent Harness Testing

Independent harness tests like this show why vendor benchmark numbers often fail to predict task-level performance for your own workflows. Builders evaluating model routers or agent harnesses can take the concrete lesson that results depend heavily on the test setup, so replicating with your own workloads matters more than leaderboard rankings. The System One versus System Two framing offers a practical lens for deciding when to route cheaply and when to pay for heavier reasoning.

Event
elvis@omarsar011 sourcesSource
Sep 26
Sep 25
Sep 24
Sep 23