NVIDIA Blackwell GPU Count Clarification
The potential quarter-trillion dollar training cost for Astra highlights unsustainable industry trends as exponential cost increases outpace modest performance gains.
Gary Marcus questions whether Jensen Huang meant 100K+ GPUs or 100K+ NVLink72 server racks, each containing 72 Blackwell GPUs. If the latter, training hardware for Astra would cost approximately a quarter trillion dollars. Rental prices might reach low tens of billions. Epoch AI Research shows the performance improvement is not off trend.
⚠️ The most important question hardly anyone is asking is whether Jensen meant 100K+ GPUs, or 100k+ NVLink72 server racks (which contain 72 Blackwell GPUs). ⚠️ From his wording, it sure looks like the latter to me. If he indeed meant 100k+ NVLink72 server racks, we can infer that the hardware to train Astra sells for something like a quarter trillion dollars. Rental prices would perhaps be in the low tens of billions. For an improvement that @EpochAIResearch shows is not off trend. One key foundational problem with this whole industry (aside from technical limits of LLMS) is that you have two trends; exponential increases in training costs, modest increases in performance. Couple that with price wars and all of this is absolutely insane. It’d be like a gas company paying exponentially more money for each extra million barrels in the midst of a massive price war. That can’t last. Nor can this. Jensen Huang @JensenHuang @ChaseLochmiller @OpenAI GPT-6 Astra, trained on ~100K+ NVIDIA Grace Blackwell NVLink72. From ChatGPT to o1 to Astra in 4 years. AGI has arrived. Congratulations @OpenAI team. 400K GPUs coming online next. 🔗 View Quoted Tweet 💬 4 🔄 5 ❤️ 12 👀 2811 📊 5 ⚡
