想把ViT模型跑到边缘FPGA上？FlexViT用硬件协同把混合层揉进统一引擎，比CPU快1.4倍，代码已经开源。
FlexViT是一种基于FPGA的可重构加速器，专为资源受限的边缘设备上的视觉Transformer推理设计。它基于SECDA-TFLite框架，通过硬件-软件协同设计将全连接和卷积层映射到统一的INT8 GEMM引擎。采用双模数据流动态切换输入和权重复用策略，并引入深度优先平铺技术避免片外部分和传输。在PYNQ-Z2 FPGA上测试，加速器执行层最高提速2.74倍，端到端推理比CPU快1.40倍。
FlexViT: A Flexible FPGA-based Accelerator for Edge Vision Transformers
Deploying Vision Transformer (ViT) models on edge platforms remains challenging due to their high computational demands and the architectural heterogeneity of modern hybrid ViT models, which incorporate both fully connected and convolutional layers. This heterogeneity leads to significant variation in tensor shapes, requiring flexible and efficient FPGA-based acceleration. In this paper, we present FlexViT, a reconfigurable FPGA accelerator for efficient ViT inference on resource-constrained edge devices. Built on the SECDA-TFLite framework, FlexViT employs a hardware-software co-design approach that maps both fully connected and convolutional layers onto a unified high-throughput INT8 GEMM engine using a runtime im2col transformation. To efficiently support diverse layer configurations, we propose a dual-mode dataflow that dynamically switches between input and weight reuse by reconfiguring the compute array at runtime. We further introduce a depth-first tiling strategy that completes accumulation in a single pass, eliminating off-chip partial-sum transfers and reducing memory bandwidth requirements. We implement FlexViT on a PYNQ-Z2 FPGA and evaluate it across a representative set of ViT models. FlexViT achieves up to 2.74x speedup on accelerator-executed layers, translating into up to 1.40x end-to-end speedup compared to CPU-only execution. The code is available at: https://github.com/gicLAB/FlexViT