RepICL：一次训练的少样本预测器可跨表征空间复用
RepICL: Reusable In-Context Prediction Across Heterogeneous Representation Spaces
一篇做少样本分类的研究：RepICL-I 在 12 个设定里全赢了 Logistic Regression，靠的是 episodic whitening 这一步，做小样本任务可以看看。
论文提出 RepShiftBench 基准，覆盖文本、图像、音频共 1,218 个编码器-数据集任务，用于检验少样本预测程序能否跨数据集和编码器复用。结果显示逐任务拟合的 Logistic Regression 在所有设定下都优于现有 in-context learner。作者随后提出 RepICL，通过 episodic whitening 对每个 episode 做规范化后再预测，其归纳变体 RepICL-I 在全部 12 个基准设定中超过 Logistic Regression。消融实验表明 episodic whitening 是主要增益来源，但并非普适有效的预处理步骤。
RepICL: Reusable In-Context Prediction Across Heterogeneous Representation Spaces
Frozen representations are widely reused for downstream classification, yet each new task typically requires fitting a new predictor. We ask whether the few-shot prediction procedure itself can instead be learned once and reused across datasets and representation spaces. To study this question, we introduce RepShiftBench, comprising 1,218 encoder--dataset tasks across text, image, and audio, with separate evaluation of generalization to unseen datasets, unseen encoders, jointly unseen datasets and encoders, and unseen modalities. The benchmark exposes a substantial gap: Logistic Regression fitted independently on each episode outperforms every evaluated in-context learner across all settings. We introduce RepICL, a meta-trained in-context learner that canonicalizes each episode through episodic whitening before prediction. Its inductive variant, RepICL-I, surpasses Logistic Regression in all 12 benchmark settings, while RepICL-T substantially outperforms existing transductive methods. Ablations identify episodic whitening as the primary source of these gains, while showing that it is not a universally beneficial preprocessing step. Across both variants, the gains concentrate on queries for which simple support prototypes favor the wrong class or provide little separation between the true class and competing classes. Transduction provides its largest additional gains when limited support coverage gives a misleading view of class separation. Together, these results demonstrate that a shared few-shot prediction procedure can generalize beyond the representation spaces observed during training.
