When Personality Meets Platform: The Moderating Role of Human-Likeness in AI User Evaluations

Abstract

This study examines how Big Five personality traits interact with the perceived human-likeness of AI to influence user evaluations of two functionally distinct AI platforms: Replika (socio-emotional companion) and ChatGPT (task-oriented assistant). Analyzing 20,000 user reviews, we found that personality traits significantly predict user ratings. Extraversion was linked to more positive evaluations for both AI platforms. In contrast, Conscientiousness and Neuroticism predict more negative evaluations. Unexpectedly, Openness also correlated with more negative responses, challenging the assumption that open users are more receptive to emotionally rich AI. Perceived human-likeness moderated these effects differently for each platform. For Replika, it enhanced the positive effects of Extraversion and mitigated the negative impacts of Openness and Neuroticism. However, for ChatGPT, increased human-likeness only reversed the negative effects of Neuroticism, leaving the impacts of Openness and Extraversion unchanged. Implications are discussed for personality-contingent anthropomorphic design.