MCP-Persona 填补了现有基准忽视个人化工具交互的空白,做智能体开发和 MCP 工具集成的团队可以直接用它来测试和优化自己的模型。
MCP-Persona 是首个专门评估 LLM 智能体在真实个人化 MCP 工具上表现的基准。它覆盖了 Reddit、小红书、飞书、Slack 等主流社交和协作平台,测试智能体与个人账户和本地数据库交互的能力。实验发现,当前最先进的智能体在处理个人化工具时表现挣扎,凸显了该基准在识别和解决这些局限性的关键作用。该基准已开源,可供开发者直接使用。
MCP-Persona: Benchmarking LLM Agents on Real-World Personal Applications via Environment Simulation
The Model Context Protocol (MCP) has emerged as a transformative standard for connecting large language models (LLMs) with external data sources and tools, and has been rapidly adopted across personal applications and development platforms. However, existing benchmarks predominantly focus on generic information-seeking tools and fail to capture the practical challenges posed by personal social applications, where tools interact with individual accounts or local databases. To bridge this critical gap, we introduce MCP-Persona, the first benchmark specifically designed for evaluating agent performance on real-world, personalized MCP tools. MCP-Persona encompasses a diverse set of widely-used applications, ranging from social media platforms like Reddit and Xiaohongshu (Rednote) to enterprise collaboration suites such as Lark (Feishu) and Slack. Our extensive experiments on various state-of-the-art (SOTA) agents demonstrate their significant struggles with personalized tool use, thereby highlighting the benchmark's crucial role in identifying and addressing these limitations. MCP-Persona is publicly available at https://github.com/wwh0411/MCP-Persona}{https://github.com/wwh0411/MCP-Persona.