做 AI 编译器或硬件优化的开发者会关心——LLM Agent 的搜索能力被高估了，实际表现受限于预训练数据分布，直接套用反馈循环可能适得其反，建议先看实验设计再决定是否采用。
该研究通过三个受控实验，系统评估了 LLM Agent 在硬件感知代码优化中的表现。研究发现，LLM 在纯黑盒优化中表现为贪婪优化器；在零样本内核生成中，提供显式输入大小信息没有可测量的效果，模型会收敛到相同的内核参数；在反馈循环优化中，CUDA 在迭代反馈下单调改进，而 TVM IR 则主动退化。结论表明，LLM 在代码优化任务中高度依赖预训练先验知识，而非提供的反馈或智能体结构。
Prior Knowledge or Search? A Study of LLM Agents in Hardware-Aware Code Optimization
LLM discovery and optimization systems are increasingly applied across domains, implementing a common propose-evaluate-revise loop. Such optimization or discovery progresses via context conditioning on received feedback from an environment. However, as modern LLM agents are increasingly complex in their structure, it is difficult to evaluate which components contribute the most, and when and how this exploration may fail. We answer these questions through three controlled experiments. Our findings: (1) In pure black-box optimization, LLMs act as greedy optimizers. (2) In zero-shot kernel generation, providing explicit input-size information has no measurable effect, models converge to the same kernel parameters regardless of size or temperature, as though the size instruction were invisible. Moreover, when tasked to perform kernel optimization for uncommon kernel sizes, performance sharply degrades regardless of the language used. (3) In feedback-loop kernel optimization, CUDA improves monotonically under iterative feedback, while TVM IR actively degrades, which demonstrates that kernel optimization degrades when models operate with low-density language. Our results conclude that LLMs in code optimization tasks highly depend on pretrained priors rather than provided feedback or agentic structure.