An ensemble approach for unsupervised anomaly and cyber attack detection

Abstract

With the increased demand for intelligence, neural networks have become an increasingly popular solution for intrusion detection systems (IDS), as their ability to learn complex patterns and behaviors makes them a suitable solution for distinguishing normal traffic from network intrusions. Hence, this paper proposes a new unsupervised IDS ensemble framework that utilizes parallel deep learning techniques based on three different anomaly detection concepts. That is, modeling the detection of point anomalies, collective anomalies, and contextual anomalies simultaneously.These detectors are trained in parallel, and the anomaly scores obtained from each detector are combined to provide the final detection decision. This ensemble approach can simultaneously consider the different types of anomalies in time series data and reduce the impact of overfitting some unsupervised anomaly detectors. Compared to supervised methods, the developed scheme reduces the overhead of manually annotated data and detects online possible novel attack data streams. The proposed ensemble model has been tested on the UNSW-NB15, DAPT 2020 and CSE-CIC-IDS 2018 datasets. Compared to baseline models, the single detectors used to constitute the ensemble model achieve better performance separately in most cases. Through three simple ensemble strategies, Vote, ‘And’ logic and ‘Or’ logic, the ensemble model exhibits improved stability, precision performance, and recall performance, respectively. This demonstrates that the proposed ensemble model can successfully combine the advantages of different unsupervised detectors, offering an advantage over other single unsupervised anomaly detection models. Moreover, the suggested method is effective for detecting various traffic anomalies caused by network intrusions that occur in the datasets.