LlMBO：用贝叶斯优化微调 LLM，算力远低于 GRPO
How Low Can You Go: Fine-Tuning LLMs with Bayesian Optimization (LlMBO)
有人在数学任务上用贝叶斯优化微调，参数不到 10 个，效果打平甚至超过 GRPO，算力显存省一大截。
Morris 等人此前提出的 TinyLoRA 证明用不到 10 个可调参数微调 LLM 也能提升准确率，而 LoRA 类方法通常需要上万参数。但 TinyLoRA 仍依赖梯度下降这类局部搜索。这篇 arXiv 论文指出在低维场景下全局优化变得可行，由此提出 LlMBO（Large language Model Bayesian Optimization），无需梯度、显存占用低、可在多设备间免同步并行，还能直接优化贪心解码准确率。在数学基准套件上的实验显示，LlMBO 对多个模型的微调收益持平或超过 GRPO，计算与内存开销只占后者的很小一部分。
How Low Can You Go: Fine-Tuning LLMs with Bayesian Optimization (LlMBO)
Recently, Morris et al. introduced TinyLoRA, a strategy for LLM fine-tuning which introduces arbitrarily few parameters, and demonstrated accuracy improvements using less than $10$ parameters rather than the ${\gtrsim 10,000}$ typically demanded by LoRA-based methods. However, that study continued to use gradient-based local search methods of the kind deployed for training multi-billion parameter LLMs in the first place. This article is predicated on the observation that, in the low dimensional setting, global optimization methods become tractable, unlocking Large language Model Bayesian Optimization (LlMBO). LlMBO is sample-efficient, gradient-free, memory-light, parallelizes across devices without synchronization, and, unlike policy-gradient methods, can optimize greedy-decoding accuracy directly. We describe numerical experiments comparing LlMBO against GRPO on fine-tuning tasks for a variety of models on a math benchmark suite which show that LlMBO can be a suitable alternative in this regime, matching or exceeding GRPO's fine tuning gains, while using a fraction of the compute and memory.
