我朋友知道有个叫NEAR的新方法,能帮脑信号图像图像检索不用试那么多遍,和以前方法比,少重复的时候效果更好了。
近期研究发现,传统脑信号到图像检索技术依赖多次试验平均,当试验次数减少时检索准确率明显下降。该团队发现问题不仅源于查询噪声,还与图像表示的画廊布局有关,于是提出NEAR框架。NEAR通过锚定神经和视觉表示来优化检索过程,在EEG、MEG、fMRI等多类数据集测试中,提升少重复场景下检索表现,如THINGS-EEG2数据集上单次试验Top-1准确率提升5.7个百分点。
Beyond Trial Averaging: Anchoring Neural and Visual Representations for Few-Repetition Brain-to-Image Retrieval
Decoding visual information from brain signals probes neural representations and enables neuro-rehabilitation and dream decoding. Recent brain-to-image retrieval approaches have achieved promising performance, typically by averaging many (up to 80) neural trials per image, requiring repeated stimulus presentation that increases latency, cost, and user burden. When only one or a few repetitions are available, the retrieval accuracy drops sharply. This drop is commonly attributed to query noise because averaging suppresses noise and increases signal stability. However, we find a non-transitive alignment pattern: the low-repetition query signal and the image representation each align with the high-repetition center, but not directly with each other. This pattern shows that query noise is only part of the problem and that gallery placement also affects retrieval. We therefore propose a neural-anchor-based retrieval (NEAR) framework that treats the high-repetition center as an anchor and approaches it from both sides: a denoiser pulls the noisy query toward the true anchor, and a small network predicts each candidate's pseudo anchor from its image and pulls the image toward it. Across four datasets spanning EEG, MEG and fMRI, NEAR consistently improved retrieval in the few-repetition regime. On THINGS-EEG2, it improved 200-way Top-1 accuracy by 5.7 and 9.3 percentage points respectively, when averaging one and four repetitions. By anchoring neural and visual representations, NEAR reduces reliance on repeated acquisition and brings neural retrieval closer to real-world deployment.