arXiv 论文：解析三种提示词适配方法在 LLM 内部的不同作用机制
Not All Drift Is a Mechanism: Tracing Prompt Adaptation in LLMs
如果你调过提示词或 prompt tuning，这篇帮你搞清楚三种方法到底在模型内部改了什么，选方法时有依据了。
该论文比较了离散提示词优化、prompt tuning 和 prefix tuning 三种方法在稠密模型和 MoE 模型内部的运作差异。研究发现适配位置决定任务信息的传递路径：优化文本主要改变响应构建，输入级提示写入内容表示，逐层前缀通过注意力保持直接控制。在 MoE 模型中，不同方法还会引发不同的专家路由模式。论文同时指出常见的表示层测量指标对通用上下文变化同样敏感，无法可靠区分真正有效的适配与匹配的安慰剂对照。
Not All Drift Is a Mechanism: Tracing Prompt Adaptation in LLMs
Prompt-based adaptation offers several ways to specialize a frozen language model, ranging from natural-language prompt optimization to learned continuous prompts. These methods are usually compared through downstream performance, although similar outputs need not arise from similar internal computations. We study how discrete prompt optimization, prompt tuning, and prefix tuning alter the processing of dense and mixture-of-experts language models. Our analysis shows that the location of the adaptation predicts how task information is carried to the output: optimized text primarily changes response construction, input-level prompts write their influence into content representations, and layer-wise prefixes maintain direct control through attention. In MoE models, these differences further induce distinct expert-routing patterns. At the same time, common representation-level measurements respond strongly to generic context changes and do not reliably distinguish useful adaptations from matched placebos. We argue that understanding prompt adaptation requires separating representation drift from the part of that drift that is aligned with, and causally used by, the model's task computation. This perspective provides practical guidance for selecting, evaluating, and interpreting prompt adaptation methods.