Event ·Official first-hand

UltraBERT-380M 单次前向多选分类,测试准确率最高达 72.1%

arXiv 技术报告 UltraBERT-JEV 研究编码器模型在单次前向传播中从多个选项里做选择、不生成文本的用法,比较了微调 UltraBERT-380M 的四种打分与概率化方案,其中一种参考了 DeepSeek-V3 的专家门控。用与 Qwen3-0.6B 相同的数据训练,Qwen3-0.6B 的 LLM-as-Jev 配方测试准确率为 67.6%,编码器达到 69.7% 至 72.1%。把选项得分从单独的 "yes" 改为 "yes" 减 "no" 带来最大提升,测试集从 69.7% 升到 71.3%,外部基准从 57.8% 升到 59.6%,而解码器方案仅 56.1%;Sigmoid 归一化再加 0.8 个点,square-root softplus 无增益。NVIDIA A100 上单条请求每次决策 29 毫秒,对比解码器的 68 毫秒,但峰值内存更高(2.8 对 2.2 GiB)。模型还用 SelfCite 改造的奖励训练的辅助头,在同一次前向中预测支撑答案的句子。

Current read

做分类或路由这类单选任务的开发者可以看看这份报告:同样数据下,UltraBERT-380M 编码器比 Qwen3-0.6B 解码器高近 5 个点,决策还快一倍多。文中 "yes" 减 "no" 打分这个细节最实用,直接能搬到自己的微调流程里。

11 sources36° AI heatLast updated 2026/11/13 07:58:46

Evidence

11 sources
Primary sourcearXiv: DeepSeek
View source
Related source 1AI21 Labs
View source
Related source 2NVIDIA AI
View source
Related source 3AWS Machine Learning Blog
View source
Related source 4The Information
View source
Related source 6rohanpaul_ai
View source
Related source 10LMSYS Org (SGLang)
View source

Conflict check

Dedup data only shows that these sources discuss the same event, not that they agree. There is no structured supporting or refuting evidence yet, so no conflict is inferred automatically.