论文证明神经网络还解不开这个经典难题
Fodor和Pylyshyn提出的系统性挑战认为，人类语言理解具有双向条件依赖（如理解"John saw Mary"就能理解"Mary saw John"），而神经网络无法解释。Lake和Baroni的元学习组合性协议声称已匹配人类系统性，但本文实验发现，该模型在分布外规则上表现困难，甚至在分布内任务中也出现非系统性行为。作者结论是Fodor和Pylyshyn的挑战仍未得到满足。
Fodor and Pylyshyn's Systematicity Challenge Still Stands
The recent successes of neural networks producing human-like language have caused significant stir in cognitive science, with many researchers arguing that classical puzzles about human cognition and challenges to artificial intelligence are being solved by neural networks. A notable case is the argument from systematicity due to Jerry Fodor and Zenon Pylyshyn, argues that humans display systematic biconditional dependencies. For example, someone can understand the sentence "John saw Mary" just in case that they understand the sentence "Mary saw John." Symbolic systems explain this systematicity of language and thought, while neural networks offer no immediate explanation. Several recent articles argue that this challenge has now been met by neural networks. In particular, Brenden Lake and Marco Baroni argue that their meta-learning for compositionality protocol matches and perhaps explains human systematicity. We demonstrate that these conclusions are premature. Among other results, we found that their model struggles to learn rules that are even slightly out of distribution compared to their training data. Furthermore, the model behaves unsystematically even on many within-distribution problems. We conclude that Fodor and Pylyshyn's challenge to neural networks remains unmet.