Design and implementation of a web camera-based natural user interface engine

Abstract

Natural User Interfaces (NUIs) are a novel way to provide Human Computer Interaction (HCI) with natural and intuitive operation interfaces, such as using human gestures and voice. This paper proposes a real-time NUI engine architecture using a web camera as a means of implementing NUI applications with an inexpensive device. The engine integrates the OpenCV library, the CUDA toolkit, and the DirectX SDK. We utilize the OpenCV library to capture video via the web camera, implement real-time image processing using Graphic Processing Unit (GPU) programming, and present the NUI applications using the DirectX SDK; for example, to implant a 3D object in a captured scene and play sounds. To verify the efficacy of our proposed NUI engine, we utilized it in the development and implementation of several mixed reality games and touch-less operation applications. Our results confirm that the methods of the engine are implemented in real time and the interactive operations are intuitive. © 2014 Springer-Verlag Berlin Heidelberg.