研究评估 GPT-4、Gemma 等模型能否模拟个人人格问卷回答

结果显示，提供用户文本并未稳定提升预测效果。模型在特质分布中段表现较好，但在分布两端往往停留在中心值，或在一端表现良好、另一端表现很差。在只给显式人格画像的条件下，GPT-4 最能将特质信息转化为一致的问卷回答，尤其在 Extraversion 和 Neuroticism 维度上。增加上下文文本长度也没有系统性降低误差：开源模型波动较大，GPT-4 相对稳定。

这项工作对“数字孪生”式个性化模拟提出了量化质疑：当前主流模型尚不能稳定复现具体个体的问卷行为，做用户模拟或社会科学实验的研究者需要谨慎对待这类应用。

Large Language Models as Digital Twins: a Distribution-Aware Questionnaire-Based Evaluation

This paper investigates whether Large Language Models can answer a personality questionnaire as a specific individual would. Given a user's social-media posts, the model completes the BFI-2-60 Big Five questionnaire as that user. The predicted responses are converted into trait scores and compared with the user's actual scores. We evaluate GPT-4, Gemma, Llama 3.1, and Mistral on the MyPersonality dataset. To determine whether predictions reflect information from the user's writing, we compare them with answers produced without user text and with values derived from general population tendencies. We measure prediction error across all users and separately for individuals with low, intermediate, and high trait scores. We also test whether an LLM can express an explicit personality profile when given only the target Big Five scores, without user text, and examine how predictions change as more text is provided. The results show that user text does not consistently improve performance. Models perform better near the middle of the trait distribution, while at the lower and upper ends they often remain close to central values or perform well at one end but poorly at the other. When given an explicit profile, GPT-4 most clearly translates trait information into consistent questionnaire responses, particularly for Extraversion and Neuroticism. Increasing context does not systematically reduce error: open-source models fluctuate more, whereas GPT-4 remains comparatively stable. Overall, the evaluated models do not consistently reproduce the questionnaire behavior of specific individuals.

This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include GPT-4, Llama 3.1, Gemma, Mistral.

Source attribution: curated from arXiv cs.AI (@Francesco Di Cursi, Chiara Boldrini, Andrea Passarella, Marco Conti). Cross-source corroboration: 2 independent sources have covered this development (including rohanpaul_ai), confirming industry-wide signal strength.