想测LLM的地理常识？这个新基准GeoBenchLLM用12个数据集考了主流模型，结论是模型越大、推理越强，GitHub上可跑。
GeoBenchLLM是一个评估大语言模型地理相关任务能力的基准，整合了12个公开数据集。该基准覆盖地理空间和时间理解两类任务，测试了多个主流LLM。结果显示，推理能力和模型尺寸对整体性能影响显著。基准代码已在GitHub公开。
GeoBenchLLM: A Comprehensive Benchmark for Evaluating LLMs on Geo-Related Tasks
In the context of geodata, existing Large Language Models have often been studied in a homogeneous setting, which has considerably limited insights into their generalization capabilities. In this paper, we present \benchName, a comprehensive benchmark for probing LLMs on geo-related tasks. We leverage a careful selection of twelve publicly available datasets from diverse geo-related tasks and domains, and evaluate a set of LLMs on geo-spatial and temporal understanding using our benchmark. Our results show that reasoning and size have a strong impact on overall performance. GeoBenchLLM is publicly available at https://github.com/Rfr2003/GeoBenchLLM.