Zing框架提升LLM社会智能:SoMBench基准与Actio推理架构

Zing: Social Mind for LLMs

精选理由

这篇论文发布了一个综合框架,让你看到当前LLM在社会智能上的短板(最高才72%),并提出Zing训练和Actio推理搭配,能显著提升表现。想了解模型怎么更懂社交规矩的可以看看。

AI 摘要

Zing是一个集成框架,用于测量、内化和落地LLM的社会智能。其评估基准SoMBench覆盖3个主要维度、17个次要维度和71个任务范式,包含284个共享场景和3481个专家验证实例。在20个代表性LLM上的评估显示最高准确率仅72.08%,17个次要维度均未达90%。Zing训练方法在5个社会认知基准上持续超越基础模型,其中Zing-27B-Stage2取得最高平均分,Zing-32B-Stage2与DeepSeek-V4-Pro竞争。Actio推理架构通过PRISM、Starling、SAGE和门控RAG四种支持,在15个模型-基准对中提升14个。

原文 · arXiv: DeepSeek

Zing: Social Mind for LLMs

As large language models move from isolated task solving toward long-term service in human environments, they require social intelligence: the ability to infer mental states, track social relations, reason over norms, and adapt behavior under context. This report presents Zhijing, an integrated framework for measuring, internalizing, and grounding social intelligence. For measurement, we introduce SoMBench, a psychology-grounded benchmark spanning 3 primary dimensions, 17 secondary dimensions, and 71 task paradigms. It controls question format, narrative perspective, and context length across 284 shared scenarios and 3,481 expert-verified instances. Evaluation of 20 representative LLMs reveals substantial headroom: the best model achieves only 72.08% overall accuracy, and none of the 17 secondary dimensions reaches the 90% near-ceiling band. For internalization, we develop Zing, a diagnosis-driven training recipe combining supervised fine-tuning, on-policy distillation, and rubric-based reinforcement learning. Across five social-cognition benchmarks, Zing consistently outperforms its base models, with Zing-27B-Stage2 achieving the best average score and Zing-32B-Stage2 remaining competitive with DeepSeek-V4-Pro. For deployment-time grounding, we build Actio, a harness-controlled inference architecture that routes four typed supports into reasoning: PRISM for procedural guidance, Starling for runtime mental-state representation, SAGE for reusable experience, and gated RAG for external social and normative knowledge. Across five base models and three benchmarks, the full harness improves 14 of 15 model-benchmark pairs and is best or tied for best in 8, demonstrating the effectiveness of typed runtime support. Together, these results show that socially intelligent LLMs require coordinated advances in evaluation, parametric internalization, and deployment-time grounding.