Latest

The takeaway is that small models can beat larger ones when given enough parallel samples and a verifier, which reframes inference-time compute as the practical lever for builders without trillion-parameter budgets. Engineers designing agent pipelines should note the coverage-versus-pass@k distinction when choosing between repeated sampling and reasoning models like o1. Stanford publishing the full lecture makes a frontier-lab curriculum freely accessible to self-taught practitioners.

shao__meng@shao__mengSource

The finding inverts common prompt-engineering instinct: adding reasoning requests degrades agent decisions while plain factual statements about rules improve them. Developers building bidding, negotiation or matching agents can copy established market-design heuristics instead of inventing new prompt tricks. Judging agents by their written plans is also misleading, because the models' choices improved even when their stated reasoning did not reflect the gains.

rohanpaul_ai@rohanpaul_aiSource

A 27B open-weights model running locally now solves multi-digit addition-in-words at near-perfect accuracy, a task GPT-4o fumbled two years ago. Practitioners evaluating small local models for arithmetic-adjacent workflows get a concrete benchmark on consumer-tier DGX Spark hardware. Comparisons across reasoning-on and reasoning-off settings make it easy to judge whether enabling reasoning is worth the latency.

The cost gap, 62x cheaper than LLM judges, makes calibrated decision models a practical replacement for expensive evaluation pipelines. It also surfaces label errors in three existing benchmarks, which anyone relying on published alignment scores should note. Thresholds do not transfer between benchmarks, so teams adopting this should fit one on roughly 10 labelled items, which lifts median F1 from 0.706 to 0.793.

elvis@omarsar03 sourcesSource

The benchmarks show where cheap small models can replace larger ones in RAG pipelines: stopping decisions and relationship filtering see large latency and cost savings with matched recall. Engineers building retrieval systems can use the published code to pick where to route between Jev, GPT-5-mini and Voyage rerank-3. The evaluation includes Chinese-language queries, useful for teams serving bilingual users.

Milvus@milvusioSource