AITOPAI热报
Papers

Stanford 提出 hinted scaling laws，可用极低算力预测基准成绩

Stanford AI Lab@StanfordAILab
Post
View the post
Why we picked it

斯坦福团队把基准题加提示变简单，再反推模型真实成绩，算力省了 5600 倍还能预测 GSM8K，做评测和选型的人可以看看这套方法。

Stanford AI Lab 的 Suze van Adrichem 等人提出 hinted scaling laws 方法，用于预测模型在基准测试上的准确率。做法是把一个基准改造成一系列加入提示（hint）的更简单版本，再联合建模模型规模 C 与提示等级 H 两个变量。结果显示，仅用 5600 倍更少的计算量，就能把 GSM8K 无提示（H=0）时的准确率预测到 3 个百分点以内。该工作由 Luke Bailey 与 rckpudi 共同主导。

Original · Stanford AI Lab

Hinted scaling laws allow you to make better benchmark accuracy forecasts! Suze van Adrichem @suze_va Can we forecast benchmark accuracy from models that score zero? We (co-leads @LukeBailey181 @rckpudi ) turn a benchmark into a series of easier hinted versions. Jointly modeling scale C and hint level H enables forecasting GSM8K acc (H=0) within 3 pts from 5600x less compute 🧵 🔗 View Quoted Tweet 💬 1 🔄 1 ❤️ 3 👀 1029 📊 2 ⚡

#scaling laws#GSM8K#Stanford AI Lab#基准评测

Keep reading

Recent stories with the same tag or section
PapersBuildersMulti-sourceSelected04:04
Stanford AI Lab's hinted scaling laws forecast GSM8K accuracy with 5600x less compute

Predicting benchmark scores without training large models changes how labs and buyers estimate model quality before committing compute budgets. The 5600x compute saving makes accuracy forecasting accessible to academic groups and startups that cannot run full-scale training runs. Teams evaluating whether to invest in a model capability can apply this method to get defensible projections at a fraction of the usual cost.

#GSM8K#Stanford AI Lab#scaling laws#benchmark
Event
Stanford AI Lab@StanfordAILab4 sourcesSource
IndustryMulti-source01:06
Chinese contentYuchen Jin 谈新 AI 实验室：多数会失败，挑战 scaling laws 者才有机会

SWE-bench 上表现很强的 LMArena 团队创始人聊了点行业实话：抄 OpenAI 作业的新实验室大多活不下去，他押注的是那些挑战 Transformer 和下一词预测范式的团队，还点破了研究员创业的通病。

#OpenAI#Anthropic#scaling laws#AI创业
Event
Yuchen Jin@Yuchenj_UW11 sourcesSource
PapersOfficialSelected00:49
Apple Introduces Tri-Modal Masked Diffusion Models

Pretraining diffusion models across three modalities from scratch departs from the dominant recipe of finetuning a unimodal checkpoint, potentially simplifying how text, image and audio generation are unified in one architecture. Researchers and engineers working on discrete diffusion or multimodal models can reuse its scaling-law findings and sampling defaults. Apple's publication puts it in direct conversation with autoregressive approaches from labs like OpenAI and Google.

#Masked Diffusion Models#Apple#multimodal#diffusion models
官方一手Apple ML ResearchSource
How-toBuildersSelected11:06
Stanford CS 312 Deep Learning Alchemy Course Goes Public

Course materials are fully open, so self-learners can rent GPUs on Modal and replicate assignments without Stanford enrollment. The exam-heavy design assumes students use AI agents for experiments, then test mastery without them — a template other educators may copy. It suits practitioners who want empirical intuition for LLM training behavior rather than another from-scratch coding course.

#CS 312#Stanford#scaling laws#Muon
shao__meng@shao__mengSource