Astra Training Compute Cost
The $1 billion training cost for Astra provides context for evaluating AI development economics compared to other major models.
Gary Marcus questioned the $1 billion training compute cost for Google's Astra model. Nina Schick confirmed Astra was trained on approximately 100,000 Blackwell GPUs with an estimated $1 billion training compute bill and 200 MW of power. She raised concerns about the lack of scientific reporting on base models and scaling factors.
Do people agree on the $1B training compute bill for Astra? (My guess was a bit higher) (re scaling: we aren’t getting data on base models and how they scale anymore; each new model includes both new base models and a lot of unrevealed harnesses, augmented data, etc, so it’s hard to know what is causing the improvements. zero science is being reported.) Nina Schick @NinaDSchick Scaling holds. Astra was trained on ~100k Blackwell GPUs with an estimated $1bn training compute bill and ~200 MW of power. What happens when the first-gigawatt cluster comes online? We have yet to see the full force of the infrastructure (hardware and energy) being built now. Obviously, there is more Intelligence coming. 🔗 View Quoted Tweet 💬 1 🔄 0 ❤️ 10 👀 3361 📊 2 ⚡
