An attention-centric transformer network for real-time human activity recognition

초록

Enhancing Human Activity Recognition (HAR) in surveillance settings is essential for supporting a wide range of applications, including security, healthcare, anomaly detection, and crowd monitoring. Despite recent progress in vision-based HAR, several challenges remain: traditional methods still rely on convolutional or recurrent operations, limiting their ability to capture global contextual information and long-range temporal dynamics. Moreover, standard attention mechanisms often fail to retain discriminative information over time, reducing their capacity to accurately represent complex spatiotemporal relationships inherent in surveillance settings. Therefore, we propose a robust framework, termed the Attention-Centric Transformer Network (ACT-Net), designed for effective HAR in indoor and outdoor environments, comprising two phases. First, the framework uses a modified Dual Vision Transformer Network (DViT-Net) as the backbone to extract discriminative features by simultaneously processing the input data. This approach allows the network to capture complex relationships within the data through dual pathways in the transformer architecture, integrating both global semantic and fine-grained local information. Similarly, we incorporate an innovative memory-based Self-attention (sAtt) module into convolutional long short-term memory (ConvLSTM), specifically designed to capture long-range dependencies in sequential data while preserving discriminative information from previous frames. The proposed approach enhances feature extraction by effectively learning local and global dependencies and emphasizes relevant information while removing irrelevant details. In addition, this study performs extensive experiments across five benchmarks, evaluating and comparing the proposed framework with various state-of-the-art techniques. The comparative experimentalanalysis shows that the proposed network delivers promising performance, consistently improving on state-of-the-art methods, with accuracies of 82.11%, 98.93%, 99.17%, 99.01%, and 98.80% on the HMDB51, UCF101, UCF50, YouTube Action, and KTH datasets, respectively. Overall, the analysis demonstrates ACT-Net's adaptability and robustness, consistently outperforming competitive networks for effective HAR. © 2026 Elsevier Inc.

키워드