Denial of Service Resilience of Authentication Systems

Introduction

Denial of service attacks on authentication systems can take two possible forms. On one hand, an attacker can prevent the network from sending the messages that it should normally transmit to its clients. On the other hand, it could force the network into sending messages it should not normally transmit. By far, the most popular DoS attack is server flooding that prevents legitimate clients from obtaining the services they request from that server.

One cause for the vulnerability to DoS in authentication systems is that the dialog between peers takes place before even a minimum pre-authentication is performed, which renders the server incapable of distinguishing legitimate from malicious traffic. Enforcing the authentication of all requests would represent a DoS attack by itself, since the server would be busy checking all digital signatures, no matter if these are valid or not. Such a method would be as dangerous as a TCP stack overflow is in case of TCP SYN attacks.

Another vulnerability is the lack of resource accounting. In this respect Spatscheck and Peterson (1999) consider that there are 3 key ingredients for protecting against DoS attacks: accounting all resources allocated to a client, detecting the moment when these resources rise above a predefined threshold and constraining the allocated resources by reducing them to a minimum level in case an attack has been detected and recovering the blocked resources.

The third vulnerability resides in the intrinsic design of the communication protocols, as described by Crosby and Wallach (2003). A new class of low-bandwidth attacks exploits the deficiencies of data structures employed in various applications. For example, hash tables and binary trees can degenerate into simple linked lists when input data is selected accordingly. Using the typical bandwidth of a dial-up modem, the authors have managed to bring a Bro server on the edge of collapsing: 6 minutes after the attack has begun, the server was ignoring 71% of traffic and was consuming its entire computational power.

Taking in consideration the global market tendency towards on-line availability, DoS attacks prove to be more dangerous than initially predicted therefore identifying them as soon as they take place is a decisive aspect. From the moment the attack has begun until it is detected and countermeasures are deployed, the targeted servers are blocked and all legitimate requests are ignored, which can result in significant financial losses. Chained attacks can occur if the communication protocol continues its dialogue with the attacker even after anomalies have been detected. The basic idea behind the so called fail-safe or fail-stop protocols is for the message-exchange to be discontinued with any client that does not follow the normal course of the protocol.

Considering the attack forms and characteristics described above, a resilient authentication system must fulfill two main requirements. First, the system must be able to detect an incoming attack as soon as possible in order to be able to respond accordingly and prevent any possible losses. Second, the system must be able to defend itself against an ongoing attack, either through its intrinsic characteristics or by deploying a set of countermeasures against the attacker. Given these requirements, we have structured this chapter into two main parts. In the first part we address the strategy and the techniques that enable an authentication system to efficiently detect DoS attacks, and their implementation into a detection engine called SSO-SENSE. In the second part we focus on the threshold puzzles concept as an a efficient way to protect against DoS attacks and analyze the case study of the SSL Handshake algorithm from both an implementation and a performance perspective.