在线推荐系统常面临用户偏好漂移和基线约束的挑战，Dri-MED 为这类问题提供了理论扎实且效果显著的解决方案，做推荐系统或在线学习的团队值得关注其算法设计。
本文提出了一种名为 Dri-MED 的算法，用于解决线性上下文随机多臂赌博机问题，其中学习者需为具有个性化偏好的用户群体提供推荐，且上下文分布随时间漂移。在实用假设下，该问题被简化为具有异方差非平稳噪声的平稳均值线性赌博机。算法还确保每次决策的平均奖励不低于基线策略，实现了与约束感知次优间隙相关的实例相关遗憾界，并具有理论保证的约束违反次数。数值实验表明，Dri-MED 显著优于忽略漂移和偏好结构的保守基线方法。
Bandits for Efficient Experimentation: Adapting to Control Group, Preferences, and Context Drifts
We consider a variant of the linear contextual stochastic multi-armed bandits, where the learner must provide recommendations to a group of users, each having its personalized preference vector, and in the presence of context distributions that are drifting over time. Under practitioner-friendly assumptions, we reduce this setting to linear bandit with stationary mean but heteroskedastic and non-stationary noise. We further study the case when the learner must ensure the mean reward of each decision must exceed that of a baseline strategy $\boldsymbolπ_0$ at each decision step. We introduce Dri-MED, an algorithm inspired from the linear version of the MED strategy, and carefully adapted to handle the non-stationary heteroskedastic noise. We show that the instance-dependent regret scales as $\tilde{\mathcal O}\left(\fracκ{\tildeΔ}d^2(\log(T)\right)$, where $\tildeΔ$ is the constraint-aware sub-optimality gap subject to policy $π_0$, with variance-aware multiplicative term $κ$ that we carefully handle using heteroskedastic regression. We further show Dri-MED enjoys $\tilde{\mathcal{O}}(d)$ expected constraint violations. Our numerical results suggest that Dri-MED significantly outperforms conservative baselines that ignores the drift and preference structure.