PerSo 基准测试：用 Reddit 说服对话评估 LLM 社交智能
PerSo: Investigating Facets of LLMs' Social Intelligence in Real-World Persuasion Scenarios
一个用 Reddit 真实说服对话测 LLM 社交智能的新基准，发现模型在行动上赢过人类、在读懂人心上还差一截，论文细节挺有意思。
PerSo 是一个基于 Reddit ChangeMyView 真实说服对话构建的基准，通过 LLM-in-the-loop 流水线生成评测任务，并经人类标注验证。该基准将社交智能拆为认知智能（推断他人心理状态）与行为智能（选择合适社交行动）两个维度分别评测。对多款 LLM 的测试显示，成绩总体随模型规模上升，但各维度表现不均。最强模型在行为任务上略超最佳人类表现，而认知任务上最佳人类参与者仍领先。
PerSo: Investigating Facets of LLMs' Social Intelligence in Real-World Persuasion Scenarios
Understanding the social intelligence of large language models (LLMs) is increasingly important for applications involving human-AI interaction and multi-agent environments. Social intelligence comprises two aspects: cognitive intelligence, the ability to infer others' mental states, and behavioral intelligence, the ability to select appropriate social actions based on these inferences to achieve social goals. Persuasion provides a natural testbed in which both aspects are simultaneously at play, yet prior benchmarks typically evaluate them in isolation using synthetic scenarios. We introduce PerSo, a benchmark that evaluates both aspects on real-world persuasion dialogues from Reddit's ChangeMyView community, using an LLM-in-the-loop pipeline to construct the evaluation tasks, validated through human annotation. Our evaluation of diverse LLMs shows that performance generally scales with model size but varies across dimensions. The strongest LLM slightly surpasses the best observed human performance on the behavioral task, while the best human participant remains ahead on the cognitive task.
