这项研究为LLM任务表示提供了更高效、更准确的方法，做模型微调或上下文学习的开发者可以直接参考其开源代码来优化自己的应用。
函数向量（FVs）是在上下文学习中提取的任务表示，可用于引导大型语言模型（LLMs）。本研究探讨了不同FV定义对指令的影响，主要关注注意力头选择和引导两个自由度。在头选择上，使用基于梯度的归因方法（如层相关性传播LRP）显著提高了效率和准确性。在FV引导方面，分布式应用比简单聚合获得了更高的准确性。代码已公开。
Fast & Faithful Function Vectors
Function vectors (FVs) are task representations elicited during in-context learning that can be used to steer Large Language Models (LLMs). However, design choices in their formulation remain underexplored. In this work, we study the impact of varying FV definitions for instructions along two degrees of freedom: attention head selection and steering. For head selection, using gradient-based attributions with Layer-wise Relevance Propagation (LRP) substantially improves efficiency as well as accuracy. For FV steering, applying it in a distributed manner yields a higher accuracy compared to simple aggregation. Our code is publicly available.