Latest

The talks show that LLM serving bottlenecks sit as much in memory architecture — CXL pooling, HBM bandwidth — as in GPU compute, a framing many benchmark writeups skip. Engineers building inference infrastructure can study the roofline analysis and LMCache MP Mode patterns for their own deployments. The event also signals a growing Korean accelerator and serving stack around FuriosaAI's RNGD and SK hynix memory.

Official accountPyTorch@PyTorchSource

The non-obvious insight is that embedding model swaps are a data-migration problem: old vectors must stay searchable until the new index is fully backfilled and validated, or search quality collapses mid-rollout. Teams running RAG or semantic search in production can follow the walkthrough to plan migrations with built-in evaluation and rollback points, comparing their own Recall numbers against the SciFact baseline before committing.

Milvus@milvusioSource

The conversation reframes GPU procurement around energy efficiency rather than raw compute, a lens that matters as power becomes the binding constraint on data-centre buildouts. Practitioners planning multi-model deployments can take Pinterest's shared-fleet approach as a template for squeezing more workloads from one hardware pool. The guidance on when to retire versus repurpose older GPUs applies directly to capacity-planning decisions.

kimmonismus@kimmonismusSource

Flawed ground-truth answers, scoring scripts that reject correct output, and postprocessing tricks can shift rankings more than real capability differences. Engineers choosing between OCR models should run task-specific evals on their own documents instead of trusting vendor-cited aggregate scores. The talk's final chapter walks through building private tests with verified answers and consistent scoring.

Hamel Husain@HamelHusainSource

The practical rule is a risk-based split: low-cost decisions can be fully delegated, but reasoning that affects others must stay with you. Teams will recognise the failure mode he names as understanding debt, where tests pass yet nobody can explain the system. Developers and engineering leads who run parallel Agent sessions daily can use his pre-task checklist and the Claude Code team's practice of asking Claude to explain each module rather than holding the whole repo in mind.

shao__meng@shao__mengSource