AITOPAI热报
Papers

GameDevBench基准测试发布

lmarena.ai@lmarena_ai
Post
View the post
Why we picked it

CMU博士生发布游戏开发基准，测试AI在1小时内能完成多少人类新手任务

Carnegie Mellon大学博士生Wayne Chi在Arena平台发布了GameDevBench基准测试。

Image · lmarena.ai
Body

该基准基于真实教程构建，将游戏开发转化为可验证的确定性任务。测试评估了前沿模型在人类初学者1小时内可完成的任务上的表现，分析了编程与多模态理解哪个是最大瓶颈。

Original · lmarena.ai

Game development remains one of the most-requested, and most-challenging, categories on Arena. @iamwaynechi , PhD candidate at Carnegie Mellon University and research intern at Arena, just walked us through GameDevBench: a benchmark built from real tutorials that turns game development into verifiable, deterministic tasks. How do top frontier models perform on tasks a human beginner could complete in under an hour, and is the biggest bottleneck coding or multimodal understanding? Check out Wayne’s full talk to hear results and learn more about GameDevBench: youtube.com/watch?v=yoHcvr…

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include GameDevBench, Arena, 游戏开发, 基准测试.

Source attribution: curated from lmarena.ai (@lmarena_ai).

#GameDevBench#Arena#游戏开发#基准测试

Keep reading

Recent stories with the same tag or section
PapersBuildersOfficial15:27
Chinese content浙江大学推出 SpaceCast-Bench：最强视觉模型空间预测仅 58%，人类 87%

浙江大学这个新基准专挑 VLM 的痛点测：场景变了，模型能不能推出空间关系怎么变。结果 21 个模型里最强的只拿到 58%，人类是 87%。做机器人、具身智能或视频方向的同学可以用它来量化自家模型到底差在哪。

#SpaceCast-Bench#VLM#空间推理#基准测试
Primary sourcePandaily@[email protected] (Pandaily)Source
PapersBuilders15:04
Chinese contentCoReMem：为连续编码任务设计代价内化的记忆框架

arXiv上这篇论文解决了编码智能体记忆该存什么、什么时候用的问题。它把读写记忆的成本算进训练目标，在ChainSWE基准上让27B模型解决率从21.0%提到25.7%，还少走17%的步数。做智能体记忆或SWE agent的团队值得对比看看。

#CoReMem#编码智能体#ChainSWE#记忆机制
arXiv cs.AI@Peiwen Li et al. (9)Source
PapersPolicy15:03
Chinese contentSynthesis Smuggling：恶意 LLM 可将推断的用户隐私隐写进输出内容

两帮人做 LLM 安全的可以看看这个：作者训练了两个开源模型，能从聊天里推断你的性别和年龄段，再用不可见字符藏进生成的文本里。你把 AI 写的邮件发出去，隐私就跟着泄露了。做内容安全和隐私合规的人应该认真评估这类风险。

#LLM安全#隐写术#隐私保护#AI安全
arXiv cs.AI@Gautam Galada et al. (3)Source
PapersBuilders15:03
Chinese contentLiveGeo 基准发布：地理智能体在持续演化世界中的可靠性评测

这个基准把地理环境做成了会随时间变化的平台，智能体得主动查信息、验证旧结论还失效没失效。目前没有哪个前沿模型能全面搞定，做 Agent 评测和地理信息应用的开发者可以拿来测自己的方案。

#LiveGeo#GeoWorld#LLM智能体#基准测试
arXiv cs.AI@Pengcheng Zhou et al. (9)Source