Design Churn as Predictor of Vulnerabilities?

1. Introduction

Security vulnerabilities are a serious threat to any organization as an exploit can cause severe monetary and reputation damage. It is essential to detect and mitigate software vulnerabilities before the software product is released. Verification and validation activities, such as security testing and code review are effective means in reducing the number of post-release vulnerabilities. However, such quality assurance is not only inexpensive, but it is also best done by engineers specifically trained in software security (McGraw, 2006). Hence, tools and techniques that can help identify components that are more likely to contain vulnerabilities can provide substantial support to the security engineers who can focus their attention and efforts on higher risk components.

One of the possible approaches to predict vulnerable components is to build statistical models using software metrics. Historically, prediction models based on software metrics are known to be very effective in defect prediction (e.g., Basili, 1996; Menzies, 2007; Nagappan, 2005). Since recently, various studies have investigated the effectiveness of vulnerability prediction models based on software metrics. As opposed to defect prediction, vulnerability prediction is much more complicated as vulnerabilities are typically few in number. Nonetheless, various studies have demonstrated the effectiveness of vulnerability prediction models based on mutually complementary set of software metrics. A number of works has investigated the predictive power of implementation-level code measures, such as size and complexity (Shin, 2011; Chowdhury, 2011). Design-level measures, such as coupling, dependencies between components, were observed to be efficient especially in terms of recall (Zimmermann, 2010; Shin, 2011). The afore-mentioned measures are static in the sense that they consider a software system at a specific point in time. Recent works have shifted their focus towards evolutionary measures, such as code churn, which is a measure of the amount of code changed within a software unit over time. The evolutionary measures could provide an even higher performance than static measures (Shin, 2011). However, there is a clear lack of research with respect to the use of evolutionary design-level measures in the domain of vulnerability prediction.