Chat-AI Models’ Understanding of Conceptual Metaphors in Poetry

Abstract

This study investigates how Chat-AI models interpret conceptual metaphors in poetry by analyzing their AI-generated responses and comparing them to human understanding. To investigate this, we conducted three experi- ments using four Chat-AI models: ChatGPT-4, ChatGPT-4o, Claude-3.5 Sonnet, and Gemini-2.0. Each experiment assessed the models’ implicit under- standing of metaphors by requiring them to either select or rate items that reflected metaphor meaning. In Experiment 1, the models outperformed human participants in a forced-choice task. In Experiments 2 and 3, the models analyzed excerpts of poetry containing conceptual metaphors. Notably, the models con- sistently distinguished between related and unrelated words or metaphors with high accuracy. However, their difficulty in interpreting personification highlights their struggle with metaphors that require deeper inferential reasoning. In our experiments of interpreting metaphors, Chat-AI models showed emergent ability to recognize, map, and evaluate metaphorical meanings. This indicates that an emergent property arises from their deep learning architectures, vast training data, and probabilistic language modeling, rather than human-like inferential reasoning. By examining AI’s performance in metaphor processing, this research contributes to broader discussions on computational creativity, language com- prehension, and the limitations of artificial intelligence in literary analysis.