在手机 NPU 上跑 LoRA 微调：不传数据也能个性化训练
First-Order LoRA Training with Phone NPUs
这篇论文教你把 LoRA 训练塞进手机 NPU，私人数据不出设备就能微调 Qwen 和 Llama 模型，还支持断点续训和散热节流，后台跑也没问题。
arXiv 论文 2610.24816 提出一套让手机 NPU 执行一阶 LoRA 训练的系统，把三个冻结权重乘法放进 NPU，训练图、适配器和优化器留在 CPU。输入梯度算子复用驻留的权重分片和反量化器，避免拷贝转置权重。系统按梯度敏感度选精度，并用逐行范围保护处理 FP16 输出。在两款 Qualcomm Snapdragon 平台上，用 Qwen 和 Llama 家族的四个 LLM 做了评估，工作负载包括夜间手机端个性化训练和公开的 LaMP-2 基准，对比了 CPU 训练器、手机 GPU 训练器和仅前向训练。
First-Order LoRA Training with Phone NPUs
On-device fine-tuning lets a large language model (LLM) learn from its user's private data without uploading it. Low-rank adaptation (LoRA) reduces the trainable parameters, but each first-order training step still multiplies by the frozen base-model weights during the forward pass, activation recomputation, and input-gradient computation. A phone's neural processing unit (NPU) already holds these weights for inference; using it for training also requires computing input gradients, which its inference interface does not support. We present a system that runs all three frozen-weight products on the NPU while keeping the training graph, adapters, and optimizer on the CPU. Its input-gradient kernels reuse the resident weight tiles and the forward dequantizer, avoiding a transposed weight copy and ensuring forward and backward use the same effective weights. To preserve usable gradients, the system selects precision by gradient sensitivity and applies a per-row range guard for half-precision (FP16) outputs. Its runtime fuses the remaining CPU operators and overlaps NPU execution with adapter computation, while resumable execution and thermal pacing let training proceed as a background job. We evaluate it on two Qualcomm Snapdragon platforms using four LLMs from the Qwen and Llama families. Our workloads cover nightly personalization with labels supplied on the phone and the public LaMP-2 benchmark; comparisons include an optimized CPU trainer, a phone-GPU trainer, forward-only training, and GPU-trained references.
