Experimental Comparison of PC Web, Smartphone Web, and Telephone Surveys in the New Technology Era

Abstract

Smartphones have become very popular globally, and smartphone ownership has overtaken conventional cell phone ownership in many countries in recent years. With this rapid rise in smartphone penetration, researchers are looking at ways to conduct web surveys using smartphones. This is particularly true of student populations where smartphone penetration is very high and web surveys are already the norm. However, researchers are raising concerns about selection biases and measurement differences between PC and smartphone respondents. Questions also remain about comparisons to traditional interviewer-administered approaches. We designed an experimental comparison between a PC web survey, a smartphone web survey and a computer-assisted telephone interviewing (CATI) survey. This study was conducted using an annual survey of students at a large university in South Korea. The CATI (interviewer-administered) survey had a higher response rate, lower margins of error, and better representation of the student population than the two web (self-administered) modes, but at a higher cost. The CATI survey also had lower rates of item nonresponse. More significant differences were found between the modes for sensitive questions than for nonsensitive ones. This suggests that CATI surveys may still have a role to play in surveys of college students, even in a country with high rates of mobile technology adoption.