Gait-Based Human Identification by Combining Shallow Convolutional Neural Network-Stacked Long Short-Term Memory and Deep Convolutional Neural Network

Abstract

Human identification using camera-based surveillance systems is a challenging research topic, especially in cases where the human face is not visible to cameras and/or when humans captured on cameras have no clear visual identity owing to environments with low-illumination. With the development of deep learning algorithms, studies that are based on the human gait using convolutional neural networks (CNNs) and long short-term memory (LSTM) have achieved promising performance for human identification. However, CNN and LSTM-based methods have the limitation of having higher loss of temporal and spatial information, respectively. In our approach, we use shallow CNN stacked with LSTM and deep CNN followed by score fusion to capture more spatial and temporal features. In addition, there have been a few studies regarding gait-based human identification based on the front and back view images of humans captured in low-illumination environments. This makes it difficult to extract conventional features, such as skeleton joints, cycle, cadence, and the lengths of walking strides. To overcome these problems, we designed our method considering the front and back view images captured in both high- and low-illumination environments. The experimental results obtained using a self-collected database and the open database of the institute of automation Chinese academy of sciences gait dataset C show that the proposed method outperforms previous methods.