Machine Learning for Intrusion Detection Systems: Recent Developments and Future Challenges

Introduction

Network security plays a crucial role in our modern world. It helps to secure our communication and information, reduce financial loss and prevent network disruption due to attacks. Particularly, due to the growth of the Internet and its importance, almost every computer in the world is connected hence they are vulnerable to attacks. We can claim that for any computer, attacks will happen sooner or later. Intrusion detection systems (IDSs) are designed to help computer systems deal with external attacks by classifying and stopping any incoming malicious traffic (Tsai et al., 2009). It is worthy to note that the IDS does not deal with internal attacks, i.e. the attacks that started from a computer within the local network. The role of the IDS is visualized in Figure 1.

Figure 1.

The IDS in a computer system

(Tsai et al., 2009) divided the IDSs into two main categories: one relies on anomaly detection and one relies on signature detection. However, regarding the recent development in the research on IDS, we propose to divide the IDSs into three main categories: signature-based approach, anomaly detection approach, and classification approach.

The signature-based IDS relies on the signatures, or rules, defined by the security experts. For instance, the experts might define that any packet with the source IP address that is the same as the destination IP address is malicious. The signature-based IDS is very popular in the early days of the Internet. It is still being used widely in the industry today (Dang, 2020a).

The anomaly detection-based IDS relies on the assumption that the benign data themselves form a common group of traffic, while the malicious traffic is an outlier and does not share their properties. The anomaly detection is visualized in Figure 2. We can see that the majority part of the data is assumed to be normal data (or benign data), and the anomalies are defined as something different from the normal data.

Figure 2.

Anomaly detection

The classification-based IDS rely on one or multiple machine learning classification algorithms (Umadevi and Marseline, 2017). To perform the classification, the algorithms require a labeled dataset (Ring et al., 2019). The required dataset is usually huge and needs to be updated frequently. The difference between anomaly detection and classification is that in the classification setting we know exactly what are attacks, while in the anomaly detection setting we just have a vague definition that the attacks are different from the benign traffic. Hence, the classification algorithms are supervised learning techniques, while the anomaly detection algorithms are unsupervised. The classification algorithms are the dominant algorithms used in the literature recently (Alqahtani et al., 2021).

In recent years, machine learning techniques have achieved a lot of success in empowering IDS (Ahmad et al., 2021). In the rest of the chapter, we will discuss the machine learning algorithms that are studied in the literature and the published datasets for training and testing the models. We then discuss the results and the limitations of the presented approaches. We draw some further research direction and conclude our paper.

Datasets

In this section, we review some of the most popular published intrusion datasets. These datasets can be used by researchers to train and test intrusion detection algorithms.