这项研究揭示了开源模型也能掌握罕见句式的语义，做 NLP 或语言学研究的开发者可以关注其训练动态与常识知识的关系，对理解模型能力边界有启发。
研究聚焦英语中罕见的配对焦点结构（如“let alone”、“much less”），构建新数据集测试模型对其语义的理解。发现中等规模开源模型能掌握这些结构的语义，但仅靠人类规模数据训练的模型失败。语义理解在训练后期出现，晚于句法知识，且与常识知识提升相关。结果表明，开源模型也能理解罕见构式，且其学习与常识知识关联。
Language Models Learn Constructional Semantics, Not To Mention Syntax: Investigating LM Understanding of Paired-Focus Constructions
Grasping the semantics of rare constructions (form-meaning pairings) has been shown to be a challenging problem that has currently only been solved by the largest LLMs. It remains an open question if open-source models have robust constructional understanding, and if so, what learning dynamics underlie the acquisition of this knowledge. Focusing on a set of rare Paired-Focus constructions in English (e.g. "let alone", "much less"), we construct a novel dataset to test their meanings using both scalar adjectival semantics and general world knowledge. Testing a wide range of models differing in parameter count, architecture, and pretraining dataset size, we find that several modestly sized models are sensitive to both the forms and the meanings of Paired-Focus constructions, though models trained on human-scale data fail at all meaning evaluations. Turning to training dynamics for a set of open-checkpoint models, we find that Paired-Focus understanding emerges later in training than Paired-Focus syntactic knowledge, and that learning of Paired-Focus semantics is correlated with gains in some domains of world knowledge. Overall, our empirical results support the conclusion that modestly sized open-source models can grasp the rare Paired-Focus constructions, and demonstrate a connection between knowledge of Paired-Focus constructions and other meaning domains.