Intrusion Detection in High-Speed Big Data Networks: A Comprehensive Approach

Abstract

In network intrusion detection research, two characteristics are generally considered vital to build efficient intrusion detection systems (IDSs) namely, optimal feature selection technique and robust classification schemes. However, an emergence of sophisticated network attacks and the advent of big data concepts in anomaly detection domain require the need to address two more significant aspects. They are concerned with employing appropriate big data computing framework and utilizing contemporary dataset to deal with ongoing advancements. Based on this need, we present a comprehensive approach to build an efficient IDS with the aim to strengthen academic anomaly detection research in real-world operational environments. The proposed system is a representative of the following four characteristics: It (i) performs optimal feature selection using branch-and-bound algorithm; (ii) employs logistic regression for classification; (iii) introduces bulk synchronous parallel processing to handle computational requirements of large-scale networks; and (iv) utilizes real-time contemporary dataset named ISCX-UNB to validate its efficacy.