Machine Learning Techniques for Network Intrusion Detection

Introduction

Current security systems offer a reasonable level of protection; however, they cannot cope with the growing complexity of computer networks and hacking techniques. They have to face continuous environmental changes both with respect to what constitutes normal behavior and abnormal behavior. As the result, security systems suffer from low detection rates (missing out serious intrusion attacks) and high false alarm rates (falsely classifying a normal connection as an attack and therefore obstructing legitimate user access to the network resources). In order to overcome such challenging problems, there has been a great number of research conducted to apply Machine Learning (ML) algorithms to achieve a generalization capability from limited training data. In recent years, ML algorithms such as Artificial Neural Network (ANN), which is generally well regarded as the universal function approximator, have demonstrated successes in many network security applications. As a flexible “model-free” approach, ANN can fit the training data very well and thus provide a low learning bias. However, they are also susceptible to the overtting problem, which can cause instability in generalization. Some models of ANN also suffer from highly demanding computation power due to their large model complexity. For an ANN model to be useful, it should perform well on the training data and generalize reliably on the unseen data. Unfortunately, learning bias, generalization variance and model complexity are somewhat incompatible, i.e. reducing one element will inevitably increase the others. Therefore, a good tradeoff of these elements should be sought.

In this chapter, an innovative ML algorithm is proposed to alleviate the limitations of currently existing IDS, enhancing the performance of intrusion detection for rare and complicated attacks. By implementing Adaptive Boosting and Semi-parametric Radial-basis-function neural networks (RBFNN), the proposed model can minimize learning bias (how well the model fits the available sample data) and generalization variance (how stable the model is for unseen instances) at an affordable cost of computation.

This chapter starts with the related works of ML approaches for Network Security domain, followed by an extensive review of ANN models. Particularly, emphasis is put on the Generalized Regression Neural Network (GRNN) and vector-quantized GRNN. These models belong to the RBFNN family which has been reported for great successes in many applications. We also provide an overview of Ensemble Learning methods in which multiple classifiers are trained to solve the same problem and their decisions are then aggregated in some manner. It is theoretically and experimentally proved that such an ensemble model can achieve superior performance compared with individual classifiers. Next, the research proposal and its features are presented. The usefulness of this model will be illustrated through its application to the Network intrusion detection problem.