这篇论文测试了GPT、Claude等LLM在真实市场机制中的表现，发现它们比人类效率低，还公开了测试框架。
研究人员通过复刻经典经济实验，用大语言模型(LLM)替代人类参与双向拍卖市场。实验显示，LLM代理的市场收敛速度较慢或无法收敛，资源分配效率低于人类市场。不同模型家族和市场角色的代理表现出显著异质性。思维链(CoT)分析发现，执行交易而非调整价格的决策与从战略考量转向紧迫感相关。
Competitive Market Behavior of LLMs
Large language models (LLMs) are increasingly deployed as economic agents, yet there is little evidence whether LLM agents are suited for participating in market mechanisms designed for humans, and whether these mechanisms deliver desired outcomes when faced with LLM agents. We address this question by replicating seminal economic experiments, replacing human subjects with LLM agents. We place agents in a double auction environment, which is a widely-used market mechanism. We check whether such a market is able to deliver an efficient allocation of resources, thereby testing a novel dimension of alignment of LLM agents -- their compatibility with a fundamental market mechanism. We find that markets populated by LLM agents exhibit slower or no convergence towards market equilibrium, thus providing less efficient allocations than markets populated by humans. We then analyze agents' individual trading decisions and find substantial heterogeneity both across model families and market roles. We also run a lexical analysis of Chain-of-Thought (CoT) traces generated by the agents. We find that the decision to execute a trade rather than continue incrementally adjusting prices is associated with a shift from strategic considerations toward urgency. We publicly release our testing framework, which can be used for future evaluations.