CivicBench：面向城市数据的 400 任务基准与 Task-State Memory 框架
CivicBench: A Benchmark and Memory Framework for Web-Based Civic Data Analytics
一篇挺有意思的论文：用 400 个真实城市数据任务测 AI 智能体，重点查结论有没有出处、证据不够时会不会老实拒答，还配了个记忆框架 TSM。
arXiv 论文提出 CivicBench，一个包含 400 个任务的城市数据分析基准，覆盖四个城市、两年时间跨度，将政务网站的服务请求、地理边界、天气和人口数据固化为可复现的冻结数据包。评测不只看计算结果，还通过可执行合约检查来源链接、覆盖范围、时空边界以及在证据不足时能否正确拒答。论文同时推出 Task-State Memory（TSM），把工具观测编译为带类型的断言，只在兼容范围内合并记录。在九组完成的模型对比中，八组的平均分差异的 95% 置信区间不含零。
CivicBench: A Benchmark and Memory Framework for Web-Based Civic Data Analytics
Municipalities publish service requests, geographic boundaries, weather observations, demographic estimates, and operational guidance through independently maintained Web portals. Civic analysis must combine these sources while preserving their schemas, source links, spatial and temporal scopes, and interpretive limits. Tool-using language agents can perform this integration, but existing task benchmarks rarely test whether a computed claim remains supported by the records used. We present CivicBench, a 400-task benchmark over four cities and two years that materializes civic Web data into frozen bundles for reproducible evaluation. Executable contracts score computation, source linkage, coverage, scope, qualified answers, and refusal when the requested conclusion is not supported. We also introduce Task-State Memory (TSM), which compiles tool observations into typed assertions, reconciles records only within compatible scopes, and retrieves a bounded state organized around unresolved constraints. Across nine completed model pairs, eight yield positive mean-score differences with 95\% semantic-contract-cluster bootstrap intervals excluding zero; the ninth interval crosses zero. The benchmark and memory framework make the records behind a claim inspectable at the point of decision.