UTP-Bench用真实数据测试GPT-5等模型，发现它们在时间缓冲、延迟感知和人群敏感规划方面与人类计划差距明显。
UTP-Bench是一个大规模旅行规划基准，整合了印度504个城市的真实旅行数据，包含景点、餐厅、住宿和多种交通网络。该基准引入了实证延迟分布和人群密度模式，用于评估旅行计划在随机条件下的表现。研究团队提出了三个评估指标：缓冲充分度评分(BAS)、人群感知时间评分(CATS)和交通延迟吸收评分(TDAS)。
UTP-Bench: Uncertainty-aware Travel Planning Benchmark
Large Language Models (LLMs) have recently demonstrated strong capabilities in automated travel itinerary generation. However, real- world travel planning is inherently uncertain: transportation delays, crowd fluctuations, and unexpected stochastic delays frequently inval- idate otherwise feasible schedules. Existing benchmarks like TravelPlanner and TripCraft assume deterministic environments, evaluating only static constraint satisfaction and ignoring whether generated plans remain robust when such uncertainties arise. To address this limitation, we introduce UTP-Bench1 , a large-scale benchmark for uncertainty-aware travel planning. The dataset integrates real-world travel data spanning 504 cities of India, including attractions, restau- rants, accommodations, and multi-modal trans- portation networks. To model realistic disrup- tions, UTP-Bench incorporates empirical delay distributions and crowd-density patterns col- lected from major cities, enabling evaluation of travel plans under stochastic conditions. We further propose three evaluation metrics, namely Buffer Adequacy Score (BAS), Crowd- Aware Timing Score (CATS), and Transport Delay Absorption Score (TDAS), which quan- tify the ability of generated itineraries to main- tain robustness against transit delays and crowd variability. Experiments with state-of-the-art LLMs like GPT-5, Qwen3, Mistral and Phi-4 re- veal substantial gaps between model-generated and human-authored plans, particularly in tem- poral buffering, delay-aware transportation scheduling, and crowd-sensitive planning.