做形式化方法或软件建模的开发者会发现，LLM 对 UML 序列图语义的理解远不如预期，依赖 LLM 生成设计文档时需谨慎验证。
一项新研究测试了 Gemini-3、GPT-5.4 和 Qwen-3.6 三个大模型对高级消息序列图（HMSC）语义的理解能力。HMSC 是 UML 序列图的基础，具有严格的形式语义。研究设计了 129 个语义任务，涵盖基本语义查询、抽象与组合、迹与标签转移系统计算。结果显示，LLM 整体准确率仅约 52%，其中基本语义理解较好（88%），但抽象与组合任务（36%）和迹与 LTS 任务（42%）表现很差。所有模型都未能理解共区域和显式因果依赖等概念。这表明 LLM 在形式化软件工程任务上仍有显著局限。
(How) Do Large Language Models Understand High-Level Message Sequence Charts?
Large Language Models (LLMs) are being employed widely to automate tasks across the software development life-cycle. It is, however, unclear whether these tasks are performed consistently with respect to the semantics of the artefacts being handled. This question is particularly under-researched concerning architectural design specification. In this paper, we address this question for High-Level Message Sequence Charts (HMSCs). These are visual models with a rigorous formal semantics that have been used for various purposes, including as a foundation for Sequence Diagrams in the Unified Modelling Language (UML). We examine whether LLMs "understand" the semantics of HMSCs by examining three LLMs (Gemini-3, GPT-5.4, and Qwen-3.6) on how they perform 129 semantic tasks ranging from querying basic semantic constructs in HMSCs (i.e., events and their ordering) to semantic-preserving abstractions and compositions, and calculating the set of traces and trace-equivalent labelled transition systems. The results show that LLMs only have a modest understanding of the formal semantics of HMSCs (ca. 52% overall accuracy), with great variability across different semantic concepts: while LLMs seem to understand the basic semantic concepts of MSCs (ca. 88% accuracy), they struggle with semantic reasoning in tasks involving abstraction and composition (ca. 36% accuracy) and traces and LTSs (ca. 42% accuracy). In particular, all three LLMs struggle with the notions of co-region and explicit causal dependencies and never employed them in semantic-preserving transformations.