做因果推断或行为建模的团队会发现,嵌入相似度直接当因果用是危险的——这篇给出了可落地的对比学习方案,还附带了加速脚本,值得直接试。
研究发现,预训练的生物医学语言模型(如BioBERT、PubMedBERT)在跨领域概念对(如“皮质醇28 ug/dL”与“股市波动性”)上给出高达0.83的余弦相似度,而正确答案应接近零。这种虚假相关性在大型行为模型(LBM)中会导致错误的因果边,进而污染下游推理。研究者提出对比学习方案:第一轮对比训练将PubMedBERT的BIOSSES相关性从0.633提升至0.828,跨领域分离度从1.05倍提升至1.63倍;第二轮BODHI方法通过挖掘知识图谱中的硬负例,将分离度提升至2.30倍,判别差距提升至+0.392。在Intel Xeon 6737P上,OpenVINO将单查询延迟从1367毫秒降至10毫秒(133倍加速),每秒可处理555个句子。研究还发现FP16在该芯片上优于INT8,并解释了原因。
Correlation Is Not Enough: Embedding Human Metadata for Individual Causal Discovery
Ask a pretrained biomedical language model whether "cortisol 28 ug/dL" and "stock-market volatility" are related, and it returns a cosine similarity of 0.83 on a scale where 1.0 means identical. The two share no mechanism. This is not a corner case: every off-the-shelf biomedical encoder we tested (BioBERT, PubMedBERT, BioM-ELECTRA) scores unrelated cross-domain pairs between 0.76 and 0.92 when the answer should be near zero. Accuracy on cross-domain discrimination is 0%. Retrieval systems survive this, because a language model downstream filters the noise. A Large Behavioural Model (LBM), a foundation model whose subject is a person rather than a sentence, does not: it reasons over a graph of a user's life and treats embedding proximity as evidence that two events are causally linked. False proximity writes a false causal edge, and everything downstream inherits the error. Here, embedding geometry is not a tuning knob; it is correctness. We report the fix. A contrastive pass over 72,034 pairs raises PubMedBERT BIOSSES correlation from 0.633 to 0.828 and within-vs-across-domain separation from 1.05x to 1.63x. A second pass, BODHI, mines hard negatives from edges absent in a biomedical knowledge graph and lifts separation to 2.30x and the discrimination gap to +0.392, at a 4.5% BIOSSES cost. On an Intel Xeon 6737P with AMX, OpenVINO cuts single-query latency from 1367 ms to 10 ms (133x) and reaches 555 sentences/sec. One finding contradicts standard advice: FP16 beats INT8 on this silicon at every serving batch size, and we explain why. The same model on a no-AMX Ice Lake instance runs 13-27x slower. We release the benchmark suite, training corpora, the BODHI generator, and the OpenVINO scripts.