Investigating Grammatical Transfer in Korean-English GPT2 Language Models
Citations

SCOPUS

0

초록

With the recent success of artificial neural language model (LMs), their language acquisition has gained much attention (Futrell et al. 2019, Hu et al. 2020, Linzen et al. 2016, Warstadt et al. 2020, Wilcox et al. 2018). This paper delves into their second language (L2) acquisition, a largely unexplored area compared to their first language (L1) learning. The primary focus is on unraveling transfer effects originating from the L1’s linguistic structures. By closely examining our LMs’ performances on English grammar tasks, this study inspects how LMs encode abstract grammatical knowledge, particularly how pre-training biases acquired from Korean (L1) influence English (L2) performances in LMs. We present exploratory experiments where LMs were first trained on the dataset representing the initial language acquisition stage, followed by fine-tuning on the second language dataset. We analyzed cross-lingual transfer effects across diverse linguistic phenomena with the BLiMP test suite. We found that L1 pre-training did not accelerate linguistic generalization in the second language. Furthermore, our results revealed significant L1-interference, where the initial language knowledge hindered the LMs' ability to acquire and apply second language rules. © 2024 KASELL All rights reserved.

키워드

GPT-2L1-interferenceneural language modelsecond language acquisitiontransfer effects
제목
Investigating Grammatical Transfer in Korean-English GPT2 Language Models
저자
Koo, KeonwooLee, JaeminPark, Myung-Kwan
DOI
10.15738/kjell.24..202406.568
발행일
2024-06
유형
Article
저널명
영어학
24
페이지
568 ~ 588