用少量梯度找方向，再用ES在子空间里搜，省资源还比GRPO-LoRA高1个点，有代码。
Hyper-ES提出用少量梯度微调获取下降方向，再在低维子空间内用CMA-ES优化逐层DARE-TIES合并系数，避免在高维参数空间中随机扰动失效。研究在Qwen2.5-Instruct和DeepSeek-R1-Distill三个骨干上测试六个数学推理数据集。结果显示，Hyper-ES相比GRPO-LoRA平均提升1%，同时减少10%的梯度更新。代码已开源在GitHub。
Hyper-ES: Effective Evolution Strategies for LLM Reasoning via Descent Direction Merging
Evolution Strategy (ES) is a promising alternative to gradient-based fine-tuning for resource-constrained Large Language Model (LLM) reasoning. However, directly applying ES to billion-parameter LLMs is highly ineffective. In such high-dimensional parameter spaces, most random perturbations are nearly orthogonal to useful update directions, leading to unstable optimization. We propose Hyper-ES, a subspace-based ES framework that avoids the weakness of ES in full-parameter search while exploiting its strength in low-dimensional optimization. Instead of asking ES to discover useful directions from random perturbations in the LLM parameter space, Hyper-ES first performs a small number of inexpensive gradient-based fine-tuning runs to obtain descent directions. Although each direction may provide only a limited improvement on its own, their span forms a compact adaptation subspace that captures useful reasoning updates. Hyper-ES then applies CMA-ES to optimize layer-wise DARE-TIES merging coefficients within this subspace, allowing ES to search over combinations of meaningful descent directions rather than over arbitrary full-model perturbations. We evaluate Hyper-ES on three Qwen2.5-Instruct and DeepSeek-R1-Distill backbones across six mathematical reasoning datasets. Results show that Hyper-ES consistently outperforms GRPO-LoRA by 1% while requiring 10% fewer space-consuming gradient updates. Code at https://github.com/kuangrepi/Hyper-ES.