ChatGPT and linguistic theory: An experimental analysis of movement

Abstract

This study examines whether ChatGPT-4o internalizes syntactic theories, focusing on Pesetsky’s (2000) account for distinct types of movement. Two experiments were conducted: the first explored intervention effects in English, Korean, and Japanese, revealing inconsistent judgments despite theoretically coherent reasoning. The second analyzed covert phrasal movement and ACD resolution in English, highlighting ChatGPT-4o's inability to consistently apply syntactic principles. These results challenge the notion that language models can serve as cognitive models of human language, emphasizing the need for further refinement in computational approaches to syntactic analysis. The findings suggest that ChatGPT-4o do not adhere to syntactic theory, highlighting limitations in LLMs.