Feasibility and Quality of a National RDD Smartphone Web Survey: Comparison With a Cell Phone CATI Survey

Abstract

Internet-enabled smartphones and wireless communication technologies are opening new ways to conduct web-based self-administered data collection for academic or nonacademic research. Considering the relative advantages of self-administration such as the low cost, overall convenience, and collection of better data about sensitive topics, survey researchers are eager to explore conducting national web surveys of the general population via smartphones, especially if they can use probability-based random-digit-dialing (RDD) sampling methods. But questions about the feasibility of such surveys remain. We conducted an experiment using national samples drawn from an RDD wireless sampling frame to compare two administration methods: a smartphone web survey using SMS (text messages) invitations and a cell phone (smartphone or feature phone) survey through computer-assisted telephone interviewing (CATI). This study was conducted using the National Survey of Smoking and Health in South Korea, a country with a very high rate of smartphone ownership. The geographic and demographic representativeness of respondents in the smartphone web mode (self-administered mode) was similar to that of the cell phone CATI survey (interviewer-administered mode), although the completion rate in the former was nearly half that of the latter. A majority of demographic variables and measures of experiences, attitudes, and perceptions of cigarettes or smoking showed significant mode differences for both unweighted and weighted estimates. The total cost of the web survey was about one fifth that of the telephone survey. The results demonstrate the potential of a smartphone web survey as a stand-alone or primary mode of data collection, if carefully designed and implemented.