An Evaluation of a Test-Driven Security Risk Analysis Approach Based on Two Industrial Case Studies

An Evaluation of a Test-Driven Security Risk Analysis Approach Based on Two Industrial Case Studies

Gencer Erdogan (SINTEF Digital, Norway), Phu H. Nguyen (SINTEF Digital, Norway), Fredrik Seehusen (SINTEF Digital, Norway), Ketil Stølen (SINTEF Digital, Norway), Jon Hofstad (PWC, Norway) and Jan Øyvind Aagedal (Equatex, Norway)
Copyright: © 2019 |Pages: 35
DOI: 10.4018/978-1-5225-6313-6.ch004

Abstract

Risk-driven testing and test-driven risk assessment are two strongly related approaches, though the latter is less explored. This chapter presents an evaluation of a test-driven security risk assessment approach to assess how useful testing is for validating and correcting security risk models. Based on the guidelines for case study research, two industrial case studies were analyzed: a multilingual financial web application and a mobile financial application. In both case studies, the testing yielded new information, which was not found in the risk assessment phase. In the first case study, new vulnerabilities were found that resulted in an update of the likelihood values of threat scenarios and risks in the risk model. New vulnerabilities were also identified and added to the risk model in the second case study. These updates led to more accurate risk models, which indicate that the testing was indeed useful for validating and correcting the risk models.
Chapter Preview
Top

Introduction

Security risk analysis is carried out in order to identify and assess security specific risks. Traditional risk analyses often rely on expert judgment for the identification of risks, their causes, as well as risk estimation in terms of likelihood and consequence. The outcome of these kinds of risk analyses is therefore dependent on the background, experience, and knowledge of the participants, which in turn reflects uncertainty regarding the validity of the results.

In order to mitigate this uncertainty, security risk analysis can be complemented by other ways of gathering information of relevance. One such approach is to combine security risk analysis with security testing, in which the testing is used to validate and correct the risk analysis results. This is referred to as test-driven security risk analysis.

The authors have developed an approach to test-driven security risk analysis, and as depicted in Figure 1, the approach is divided into three phases. Phase 1 expects a description of the target of evaluation. Then, based on this description, the security risk assessment is planned and carried out. The output of Phase 1 is security risk models, which is used as input to Phase 2. In Phase 2, security tests are identified based on the risk models and executed. The output of Phase 2 is security test results, which is used as input to the third and final phase. In the third phase, the risk models are validated and corrected with respect to the security test results.

Figure 1.

Overview of the test-driven security risk analysis approach

978-1-5225-6313-6.ch004.f01
Source: Authors' work

Although strongly related, it is important to note that test-driven risk analysis is different from the more common combination of risk analysis and testing, which is referred to as risk-driven (or risk-based) testing. The purpose of risk-driven testing is to makes use of risk assessment within the testing process to support risk-driven test planning, risk-driven test design and implementation, and risk-driven test reporting. Großmann and Seehusen (2015) provide a detailed explanation of these two approaches by combining the well-known and widely used standards ISO 31000 (ISO, 2009) and ISO/IEC/IEEE 29119 (ISO, 2013a), with a focus on security.

In this chapter, the authors present an evaluation of the test-driven security risk analysis approach based on two industrial case studies. The objective of the case studies was to assess how useful testing is for validating and correcting security risk models. The basis of the evaluation is to compare the risk models produced before and after testing. That is, the authors compare the difference in risk models produced in Phase 1 with the updated risk models produced in Phase 3.

The first case study was carried out between March 2011 and July 2011, while the second case study was carried out between June 2012 and January 2013. In the first case study the authors analyzed a multilingual financial Web application, and in the second case study the authors analyzed a mobile financial application. The systems analyzed in both case studies serve as the backbone for the system owner's business goals and are used by many users every day. The system owners, which are also the customers that commissioned the case studies, required full confidentiality. The results presented in this chapter are therefore limited to the experiences from applying the test-driven security risk analysis approach.

The reminder of the chapter is structured as follows. First, the chapter describes the background concepts relevant for this chapter and then describes the test-driven security risk analysis approach, followed by a description of the research method. Then, the chapter gives an overview of the two case studies as well as the results obtained which are the basis of the evaluation. Then, the chapter provides a discussion of the results with respect to three research questions and an overall hypothesis defined as part of the research method. Finally, the chapter discusses related work and highlights the key findings before providing the conclusion.

Top

Background

This section provides the main background concepts in security risk assessment, security testing, and their combination. The key terms and definitions are summarized at the end of this chapter.

Key Terms in this Chapter

Test Case: A set of preconditions, inputs (including actions, where applicable), and expected results, developed to drive the execution of a test item to meet test objectives, including correct implementation, error identification, checking quality, and other valued information. The terms test case and test are sometimes used interchangeably.

Security Risk Assessment: The process of risk identification, risk estimation, and risk evaluation specialized towards security.

Security: The preservation of confidentiality, integrity, and availability of information.

Risk-Driven Security Testing: Security testing that makes use of security risk assessment within the security testing process to support risk-driven test planning, risk-driven test design and implementation, and risk-driven test reporting.

Test Procedure: A sequence of test cases in execution order, and any associated actions that may be required to set up the initial preconditions and any wrap up activities post execution.

Test-Driven Security Risk Analysis: Security risk analysis that makes use of security testing within the security risk analysis to validate and correct risk models.

Security Testing: The process of software testing to check whether a system meets its specified security requirements.

Complete Chapter List

Search this Book:
Reset