Town CEO Jean-Denis Greze Explains Why He Shut Down a Profitable AI Tax Startup

The talk links open model releases to concrete security practices like independent auditing and dynamic safeguards, rather than treating openness and safety as opposing goals. Researchers and ML engineers can follow the named stack — PyTorch, TorchTitan and vLLM — to reproduce or audit open models themselves. Cohere's stance is notable as a US-listed enterprise lab that publicly backs open science. origin

官方账号PyTorch@PyTorchSource

Hardware vendors gaining first-class PyTorch and vLLM support means open-source inference stacks are extending beyond NVIDIA GPUs to dedicated AI accelerators. Teams building multi-backend deployments should watch llm-d and the out-of-tree accelerator CI/CD work, which lower the cost of adding new hardware backends. Spyre's path contrasts with closed proprietary stacks like NVIDIA's CUDA ecosystem.

官方账号PyTorch@PyTorch2 sourcesSource

The bonding method determines the outcome: 3DS adds capacity at DRAM speeds, while hybrid bonding delivers bandwidth much closer to SRAM, deciding how few chips are needed to serve large models fast. Inference infrastructure teams evaluating Groq-class LPUs versus Nvidia GPUs, and anyone tracking Cerebras, which plans DRAM stacking on its CS-6 wafer, should watch which route ships first. Capacity, not bandwidth, is emerging as the binding constraint across SRAM-first architectures.

rohanpaul_ai@rohanpaul_ai11 sourcesSource

The cost crossover at 6.8K tokens/s/chip reframes the Ascend-versus-Blackwell debate from raw benchmark losses to workload-dependent economics, suggesting DeepSeek could run its API on a sub-US$1B Blackwell cluster while pivoting post-training to Ascend. Practitioners tracking inference infrastructure or Huawei's CANN ecosystem get a rare concrete per-chip price and throughput comparison. The proposed 128-chip Ascend supernode topology has no Nvidia equivalent, which matters for anyone planning large-scale decode deployments.

Teortaxes@teortaxesTex11 sourcesSource

Meta framing its entire training and inference toolchain as PyTorch-native signals that the framework's governance and roadmap will keep tracking Meta's internal needs, which affects anyone planning long-term infrastructure bets. Practitioners building on TorchTitan or ExecuTorch can use this talk to anticipate which components get first-class investment. Hardware teams seeking accelerator support will find the partnership direction relevant.

官方账号PyTorch@PyTorchSource

Specialising silicon per inference phase sidesteps the compromise a single chip type must make, and avoiding HBM removes exposure to the tightest supply constraint in AI hardware. Teams running high-volume generation workloads can compare this two-chip endpoint against all-GPU serving on cost and tokens-per-second. It also signals a broader shift toward heterogeneous inference stacks that infrastructure buyers should track.

rohanpaul_ai@rohanpaul_aiSource