What Do We Know About Buffer Overflow Detection?: A Survey on Techniques to Detect A Persistent Vulnerability

What Do We Know About Buffer Overflow Detection?: A Survey on Techniques to Detect A Persistent Vulnerability

Marcos Lordello Chaim (School of Arts, Sciences and Humanities, University of Sao Paulo, Sao Paulo, Brazil), Daniel Soares Santos (Institute of Mathematical Sciences and Computing, University of Sao Paulo, São Carlos, Brazil) and Daniela Soares Cruzes (Department of Software Engineering, Safety & Security, SINTEF Digital, Trondheim, Norway)
DOI: 10.4018/IJSSSP.2018070101


Buffer overflow (BO) is a well-known and widely exploited security vulnerability. Despite the extensive body of research, BO is still a threat menacing security-critical applications. The authors present a comprehensive systematic review on techniques intended to detecting BO vulnerabilities before releasing a software to production. They found that most of the studies addresses several vulnerabilities or memory errors, being not specific to BO detection. The authors organized them in seven categories: program analysis, testing, computational intelligence, symbolic execution, models, and code inspection. Program analysis, testing and code inspection techniques are available for use by the practitioner. However, program analysis adoption is hindered by the high number of false alarms; testing is broadly used but in ad hoc manner; and code inspection can be used in practice provided it is added as a task of the software development process. New techniques combining object code analysis with techniques from different categories seem a promising research avenue towards practical BO detection.
Article Preview

1. Introduction

Unintended parties have always tried to access sensitive data stored in software systems. Interconnected computers, though, have multiplied exponentially the harm caused by a breach of security. One of the first largely publicized attacks compromising a great number of interconnected computers was caused by the Morris (or the Internet) worm of 1988. It exploited particulars of the then common DEC VAX machines, the 4 BSD operating system, and the figerd service. However, this exploitation could only be perpetrated by taking advantage of a vulnerability known as buffer overflow or overrun (Erlingsson, 2007).

Buffer overflow (BO) occurs in programs written in languages that do not control the boundaries of arrays during run-time, for instance, FORTRAN, C, and C++ (Spafford, 2003; Viega, Bloch, Kohno, & McGraw, 2000). The goal is to overwrite memory positions to introduce malicious data. The malicious data contain an attack payload, i.e., the malicious code itself, and also a pointer to it. The successful attack does not only copy the malicious data into the memory but also alter the program execution flow to invoke the attack payload. A BO exploitation allows an attacker to take control of the system. The Morris worm was able to replicate itself because it could attack other computer systems from those whose control were surrendered to it (Spafford, 2003). Another outcome is the crashing of the system, which may lead to Denial of Service (DoS) attacks.

Because BO is associated with the very beginning of the Internet, much research effort has been devoted to address it. One solution is the adoption of memory safe languages such as Java and C# (Horstmann & Cornell, 2005; Drayton, Albahari, & Merrill, 2001), among others. These languages provide mechanisms to control the unintended access to memory positions. For instance, Java and C# check the boundaries of an array when a position beyond the limits are accessed. In these cases, an exception is thrown and the program can take an action to handle it. Another approach is to enforce memory checks in programs written in C/C++ or to add extra code to monitor whether an attack has occurred (Nagarakatte, Zhao, Martin, & Zdancewic, 2009; Ding, He, Wu, Miller, & Criswell, 2012).

With those solutions put in place, BO should be a thing of the past. However, BO exploitations are still common. In July 2012, multiple buffer overflow vulnerabilities were found in the firmware used for VeriFone point-of-sale devices. By exploiting these vulnerabilities, hackers were able to control the terminal and log information input by customers, such as PIN numbers and the magnetic stripe data of a bank card (Constantin, 2012). Among the top security risks for mobile applications (app) tallied in 2016 by the OWASP1, “client code quality issues” ranks position #7 (OWASP, 2017). This risk category encompasses vulnerabilities such as buffer overflow, format string, use of insecure or wrong APIs, and insecure language constructs.

Why is an old security issue so persistent? The reason is because C and C++ are still used. Many web and mobile applications rely on application programming interfaces (API) written in “old” languages as C and C++ (Sadeghi, Bagheri, Garcia, & Malek, 2017). If an API is vulnerable, the whole application is as well. Moreover, these languages are still used to develop new applications (Teixeira et al., 2015). C is the second most used language after Java to develop Internet of Things (IoT) applications, according to a recent survey conducted by the Eclipse IoT Working Group and other organizations (Skerret, 2017). The threat to IoT devices is particularly disturbing because they are supposed to take care of people’s intimate activities (Padmanabhuni & Tan, 2015b).

Complete Article List

Search this Journal:
Open Access Articles: Forthcoming
Volume 11: 2 Issues (2020): Forthcoming, Available for Pre-Order
Volume 10: 2 Issues (2019): 1 Released, 1 Forthcoming
Volume 9: 4 Issues (2018)
View Complete Journal Contents Listing