arXiv 新论文：用激活碎片化几何检测 LLM 被压制的知识
The Geometry of Ignorance: Detecting Suppressed Knowledge in LLMs through Activation Fragmentation
有人用匿名改写前后的隐状态几何差异，判断模型是真不知道还是憋着不说，代码也开源了，做遗忘评测的可以看看。
论文提出通过 anon-vectors 的几何一致性来判断模型是真无知还是压制了已知知识。实验覆盖来自 3 个模型家族的 10 个指令微调 LLM，碎片化指标均能区分已知与未知概念。在遗忘后评估中，该几何特征与 4 类攻击、5 种 unlearning 算法下的对抗恢复结果相关，且层级分析发现不同攻击在不同深度利用该几何结构。该方法无需实际攻击即可作为 unlearning 鲁棒性的单阶段代理指标。
The Geometry of Ignorance: Detecting Suppressed Knowledge in LLMs through Activation Fragmentation
Evaluating whether a large language model (LLM) genuinely knows a concept remains a fundamental problem, since behavioral metrics can be fooled by suppression: a model that refuses to answer may still harbor recoverable knowledge. The framework measures the geometric coherence of \textit{anon-vectors}, the differences in hidden states between a concept-specific query and its anonymized paraphrase, and shows that genuine knowledge forms a tight, low-entropy cluster while ignorance forces geometric fragmentation. Across ten instruction-tuned LLMs from three model families, all fragmentation metrics reliably separate known from unknown concepts, and post-unlearning, fragmentation geometry correlates with adversarial recovery across four attack types and five unlearning algorithms, though the direction of that relationship is shown to depend on attack methodology. Furthermore, a layer-resolved analysis reveals that different adversarial mechanisms exploit this geometry at varying depths, with high attack-specific correlations that are invisible to behavioral evaluation. Together, these results constitute a single-phase, attack-free proxy for unlearning robustness that captures aspects of parametric knowledge inaccessible to output-level metrics. A reference implementation is available at https://github.com/AdarPeleg/geometry-of-ignorance