偏好学习是许多决策系统的核心，但标注成本高。Info-Synth 解决了传统主动学习计算昂贵和反馈不可靠的问题，做推荐系统、人机交互或机器人调优的团队可以直接参考该方法来降低数据成本。
该论文提出了一种名为 Info-Synth 的主动查询合成框架，用于高效学习用户偏好。传统主动学习方法依赖池评估，计算成本高，且忽略了查询反馈的可靠性差异（如相似或完全不同物品的比较会产生模糊反馈）。Info-Synth 通过连续空间中的互信息最大化目标生成最优查询，并引入置信度感知响应模型处理模糊比较。此外，论文还提出了 Pair M-dist 和 Pair Opt-dist 两种策略，用于在有限查询池中高效选择查询。实验在合成偏好学习、文本摘要和机器人控制器调优等任务中验证了该方法的有效性。
Active Query Synthesis for Preference Learning
Efficient learning of user preferences is crucial for many modern decision making systems but typically requires costly labeled data. Active learning reduces this cost, yet standard methods are computationally expensive due to pool-based evaluation. Further, most methods assume all query feedback is equally reliable, ignoring that pairwise queries between nearly identical or entirely dissimilar items yield ambiguous, low-confidence responses. To address the issue of feedback reliability, we introduce a novel confidence aware response model that explicitly accounts for these ambiguous comparisons. To overcome the computational bottleneck of pool-based evaluation, we propose an active query synthesis framework, Info-Synth that generates optimal queries by maximizing a mutual information-based objective within a continuous space. Moreover, we propose two strategies, Pair M-dist and Pair Opt-dist, that extend Info-Synth to select effective queries even when restricted to finite query pools. We demonstrate our framework's versatility and performance across synthetic preference learning, constrained text summary datasets, and subjective, continuous-space controller gain tuning for a simulated mobile robot.