Tencent Hunyuan Introduces WebCraftBench for Evaluating Web-Building Agents
Chinese labs are now publishing their own agent evaluation benchmarks; WebCraftBench targets the same web-coding agent space that US benchmarks like SWE-bench style evaluations test, giving builders a new reference point.
Tencent Hunyuan (Tencent) released WebCraftBench, a benchmark that has agents use the live app they built rather than trusting their completion claims. It uses coverage-guided exploration to find untested paths, then scores aesthetics, usability and whether the original request was met. On 197 human-validated pairs, the benchmark matches human preference 85.3% of the time. The accompanying paper is on arXiv as 2609.15387.
The AI says the site is done. Then the homepage errors, the buttons overlap, and it added a login flow you never asked for. 🙃 Introduces WebCraftBench: agents actually use the live app, coverage-guided exploration finds what never got reached, then we score aesthetics, usability, and whether the original request was met. On 197 human-validated pairs, it matches human preference 85.3% of the time. Paper: arxiv.org/abs/2609.15387 Z 💬 2 🔄 2 ❤️ 47 👀 2213 📊 13 ⚡
- lmarena.ai09-23 20:30原文
