研究探讨嵌入偏倚对条件独立性检验的影响及修正方法
Embedding-Bias in Conditional Independence Testing
做因果推断或用文本嵌入做检验的值得看看：论文解释了为什么用嵌入代替原始变量会出偏，还给了个能控制偏倚的稳健检验。
论文研究在文本或图像条件下做条件独立性检验时，用嵌入 ψ(Z) 代替 Z 本身带来的偏倚问题。作者指出当嵌入丢弃的信息与 X 或 Y 相关时，原假设下的拒绝概率可能趋于 1。针对受 Generalised Covariance Measure 启发的残差相关检验，论文推导出偏倚等于被丢弃部分的相关性与两个偏 R² 几何平均的乘积，并据此提出一个带容差参数的稳健检验。在合成数据与文本嵌入实验中，该稳健检验的名义水平大体保持；在语言模型生成的文本上，即使采用生成器自身的内部状态作为嵌入，嵌入检验仍存在偏倚。
Embedding-Bias in Conditional Independence Testing
To test conditional independence of $X$ and $Y$ given a text or an image $Z$, one conditions on an embedding $ψ(Z)$ in place of $Z$. The embedded test is valid if $Z$ is independent of $X$ or of $Y$ given $ψ(Z)$, which cannot be confirmed from data, and when this fails, the rejection probability under the null hypothesis can tend to one. We study this failure, and show that focusing on a specific form of dependence relaxes what the embedding must retain. For a residual correlation test inspired by the Generalised Covariance Measure, validity only requires that the parts of $\mathbb{E}[X \mid Z]$ and $\mathbb{E}[Y \mid Z]$ missed by $\mathbb{E}[X \mid ψ(Z)]$ and $\mathbb{E}[Y \mid ψ(Z)]$ are uncorrelated. Otherwise, we treat the discarded information as an omitted variable. Under the null hypothesis, the bias equals the absolute correlation of the missed parts times the geometric mean of two partial $R^2$ values. This identity yields a robust test valid under a declared tolerance for the geometric mean, which, like a sensitivity parameter, is not identified from the data. On synthetic data and text embeddings, the robust test holds its level approximately. On text generated by a language model, under an exact null hypothesis, every embedding, even the generator's own states, biases the embedded test.
