这篇论文教你如何用LLM代理模拟A/B测试，比传统方法快且成本低，准确率高达90%
研究人员提出了一种基于LLM的模拟框架，通过基于真实用户行为信号的数据驱动人格条件化代理来预测A/B测试结果。该框架使用匿名行为数据构建代理，包括活动模式、参与信号和推断的人口统计信息。在包含40个A/B测试的基准测试中，最佳配置实现了0.75-0.90的方向准确率，具体取决于测试指标类型。
Data-Driven Persona-Conditioned Agents for A/B Test Simulation
A/B testing is the gold standard for evaluating product changes, but each experiment requires real user traffic, engineering effort, and weeks of measurement. We propose a simulation framework that predicts A/B test outcomes using LLM-powered agents conditioned on data-driven personas grounded in real user behavioral signals. Unlike prior work that relies on synthetic or rule-based personas, our agents are constructed from anonymized behavioral data-activity patterns, engagement signals, and inferred demographics-enabling more faithful population modeling. We frame A/B test simulation as a structured question task and systematically study (i) question design formats, (ii) the impact of persona data source and domain alignment, (iii) the trade-off between per-persona behavioral depth and population diversity, and (iv) efficient population subsampling. On a benchmark of 40 A/B tests spanning two metric types, our best configuration achieves 0.75-0.90 directional accuracy depending on the test metric, demonstrating that data-driven personas are a viable path toward fast, low-cost experiment pre-screening.