SchemaGUI:可控制GUI生成评估的模板基准

SchemaGUI: A Schema-Driven Benchmark for Controllable GUI Generation Evaluation

精选理由

SchemaGUI提供了一个新的基准,用于评估可控制GUI生成,特别是对于LLMs在几何空间控制和布局复杂性方面的表现进行了深入分析,对于想要了解GUI生成领域最新进展的人来说是个好资源。

AI 摘要

SchemaGUI是一个基于模板的基准,用于可控制GUI生成评估。通过从参数化界面模式合成成对的自然语言指令和确定性的函数调用引用,SchemaGUI可以在几秒钟内生成数千个确定性的标注任务,无需人工标注。在六个代表性双语场景中,对每个场景和语言评估了1000个实例,包括Qwen3.5系列、Qwen3-Coder-30B和DeepSeek-R1等五个主流模型。分析揭示了三个关键发现:精确的几何空间控制仍然是重要瓶颈;从4B扩展到27B的Qwen3.5提高了Schema可行性从91.56%到99.63%,但几何分数仅略有提高(从67.05%到75.30%);生成难度高度敏感于布局复杂性,当前LLMs擅长简单顺序排列,但在密集网格和多区域组合中存在严重的坐标漂移;思考模式增加令牌消耗,同时通常降低GUI分数,特别是对于较小的模型。

原文 · arXiv: DeepSeek

SchemaGUI: A Schema-Driven Benchmark for Controllable GUI Generation Evaluation

Large language models (LLMs) have demonstrated strong potential in graphical user interface (GUI) generation, but reliable evaluation remains challenging due to uncontrolled data distributions, noisy annotations, and limited layout scenario coverage. To address this, we propose SchemaGUI, a template-based benchmark for controllable GUI generation evaluation. By synthesizing paired natural language instructions and deterministic function-call references from parameterized interface schemas, SchemaGUI can generate thousands of deterministically annotated tasks in seconds without human labeling. Based on 1,000 evaluated instances per scenario and language across six representative bilingual scenarios, we benchmark five mainstream models, including the Qwen3.5 family, Qwen3-Coder-30B, and DeepSeek-R1. Our extensive analysis reveals three key insights. First, precise geometric spatial control remains an important bottleneck; while scaling Qwen3.5 from 4B to 27B improves Schema Feasibility from 91.56% to 99.63%, the Geometry score improves more modestly (from 67.05% to 75.30%). Second, generation difficulty is highly sensitive to layout complexity, with current LLMs excelling at simple sequential arrangements but suffering severe coordinate drift in dense grids and multi-region compositions. Third, thinking mode increases token consumption while generally reducing GUI Score, particularly for smaller models.