NVIDIA 推出 X-Token：跨分词器知识蒸馏，在 Llama-3.2-1B 上超越 GOLD 3.82 平均分
NVIDIA Introduces X-Token: Projection-Guided Cross-Tokenizer KD That Outperforms GOLD by +3.82 Average Points on Llama-3.2-1B
X-Token 解决了知识蒸馏中分词器不匹配的痛点，做模型压缩或边缘部署的团队可以直接用这个思路提升小模型推理能力，值得关注。
NVIDIA 提出 X-Token，一种投影引导的跨分词器知识蒸馏（KD）方法，解决了 GOLD 方法中的两个结构性缺陷。
在 Llama-3.2-1B 模型上，X-Token 将 GSM8k 准确率从 2.
56% 提升至 15.54%，并在多个基准测试中平均超越 GOLD 3.
82 分。该方法通过投影层对齐教师和学生模型的分词器，实现了更有效的知识迁移。
X-Token 为小模型蒸馏大模型知识提供了新思路，尤其适用于资源受限场景。
NVIDIA Introduces X-Token: Projection-Guided Cross-Tokenizer KD That Outperforms GOLD by +3.82 Average Points on Llama-3.2-1B
NVIDIA's X-Token fixes two structural failures in GOLD and improves GSM8k accuracy from 2.56 to 15.54 The post NVIDIA Introduces X-Token: Projection-Guided Cross-Tokenizer KD That Outperforms GOLD by +3.82 Average Points on Llama-3.2-1B appeared first on MarkTechPost .
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include 知识蒸馏, NVIDIA, Llama-3.2-1B, GSM8k.
Primary source verification: published directly by marktechpost with first-party authority. Cross-source corroboration: 7 independent sources have covered this development (including arXiv: OpenAI, IT之家, NVIDIA AI), confirming industry-wide signal strength.
