手把手教你用 TileGym 在 Colab 上跑 cuTile 和 Triton，从向量加法到 Flash Attention，每个步骤都和 PyTorch 对照，适合想深入 GPU 编程的人。
本教程使用 TileGym 在 Colab 工作流中探索 NVIDIA 的 tile-based GPU 编程，覆盖 CUDA 环境检测、cuTile 后端及在缺乏 cuTile 栈时回退到 Triton。核心思想是操作整个数据块而非单线程，包括加载、计算和存储。实现了向量加法、融合 GELU、逐行 softmax、分块矩阵乘法和 Flash Attention，每个实现均与 PyTorch 对比验证。
A Coding Guide to NVIDIA’s Tile-Based GPU Programming: From cuTile and Triton Kernels to Flash Attention
In this tutorial, we explore NVIDIA tile-based GPU programming with TileGym, building a Colab workflow that runs across different hardware. We probe the CUDA environment, try the real cuTile backend, and fall back to Triton when standard Colab GPUs lack the cuTile stack. We learn the core tile idea: operate on whole data tiles instead of single threads, then load, compute, and store them. We implement vector addition, fused GELU, row-wise softmax, tiled matrix multiplication, and flash attention, checking each against PyTorch. The post A Coding Guide to NVIDIA’s Tile-Based GPU Programming: From cuTile and Triton Kernels to Flash Attention appeared first on MarkTechPost .