Latest

Every published item, in order · 29223 items
Nov 9
Tomorrow 08:00, the daily lands in your inboxTelegram channel or email. Same screening as this page, no ads.
ModelsBuildersMulti-sourceSelected07:54
Artificial Analysis Decision Index v2 ranks top decision models

The heavier private-dataset weighting and confidence-aware scoring make it harder for models to game public benchmarks, shifting how leaderboard positions are earned. Teams selecting a decision model should weigh the 60x cost spread — Celeris-1.1 Decision sits on the Pareto frontier at US$14 per 1M tasks versus Drex Max v1.1 at US$102 — since token price alone understates real task costs.

Event
Official accountArtificial Analysis@ArtificialAnlysSource
PapersBuildersSelected07:42
Turtle Evolves AppWorld Tasks Into Harder Benchmarks

Saturated benchmarks lose their ability to separate models, and TURTLE offers a cheaper path than building new tests from scratch. Practitioners evaluating tool-use or agent models can apply the evolution pipeline to refresh existing benchmarks against their own target model, keeping failure signal alive. The 52.5% versus 19.8% hardness rating quantifies how much headroom task mutation adds.

rohanpaul_ai@rohanpaul_aiSource
PapersBuildersSelected05:30
Amazon paper finds tax and legal benchmarks misgrade correct AI answers

Benchmark scores on rule-following tasks may reflect grading bugs rather than model weakness, so leaderboard gaps deserve scrutiny before drawing conclusions. Teams evaluating models on tax, legal or compliance tasks should read scorer code and manually inspect a sample of failed answers before trusting low numbers. The paper's audit method, tracing references to source rules, offers a reusable checklist for anyone relying on public benchmarks.

rohanpaul_ai@rohanpaul_aiSource
PapersBuildersSelected71°05:07
Yann LeCun Clarifies the Difference Between JEPA and JEA

The real takeaway is a taxonomy: JEA methods like DINO and Barlow Twins suit symmetric view-matching, while JEPA's predictor targets corrupted-input or temporal prediction, with an action variable making it a world model. Researchers comparing self-supervised approaches or building world models get a concise map of which loss function fits which task.

Official accountYann LeCun@ylecunSource

Only the latest 2000 items are listed; for older ones see the daily archive