Article Preview
TopIntroduction
Today, an ever-growing number of sensitive transactions of data take place on-line (e.g., e-government, internet banking and e-commerce), and cyber crime has become prevalent. One of the consequences of this is that the cybersecurity of information systems has become an increasing concern. Assessing the level of risk posed by specific events is an area of ongoing interest for most (if not all) organisations, leading to a requirement for scientific methods of validating the cybersecurity of software systems.
This special issue asks two questions:
In the real world, the subjective opinions of cybersecurity experts are used to assess the security of software systems in their design stage. Indeed, this is often the only way to make such assessment. However, the human elements involved introduce the potential for inconsistency both between experts, and within the experts themselves. In this paper we show how the opinions of experts can be elicited and measured, and how we can ameliorate (and measure) the effects of their inherent variation through aggregation to produce a consistent and repeatable assessment.
We contend that ‘indisputable evidence’ of security is not a practical concept in the real-world, as no system can be guaranteed to be without security risks for any length of time. Furthermore, we argue that independent, measured, ‘proven’ expert opinion should be used to verify and validate software systems. Repeated successful assessments by cybersecurity experts provide the historical scientific evidence that their opinions are a valid and proven method of assuring the security of software systems. The job of security practitioners is to make an informed judgement as to whether the system is secured to an appropriate degree for the threat environment it faces.
The work described in this paper addresses two key topics of interest, Measuring Human Factors in Security – In the method we present, human experts are used to validate software systems. Perceptions of security vary both between experts, and within an individual expert. Our method allows us to explicitly measure this variation, and produce an assessment that accounts for it. Quantitative Security Management – The outcome of the proposed method is a quantification of expert opinion of the security of a system, including a measurement of uncertainty. These values can be used to make decisions regarding the implementation of a software system, and the management of its security framework.
In the proposed method we use two different types of survey to elicit the opinions of a group of security experts. The first involves ranking a series of technical attacks on a system in order of how difficult they are to carry out undetected by the system or its operators. The second requires experts to rate components of attacks in terms of aspects which are thought to contribute to their overall difficulty. In practice, a system is only as secure as its weakest element, i.e., the easiest way in. Identifying which are the weakest aspects of a system, i.e., the ‘easiest’ ways of attacking it, is thus a highly relevant component of system security assessment, though obviously it does not provide all of the answers.
We have applied the method to measure variation within a set of thirty nine highly experienced expert practitioners including system and software architects, security consultants, penetration testers, vulnerability researchers and specialist systems evaluators.
This paper shows how we are able to use expert opinion to produce a consistent assessment, and identify outliers. We also discuss the meaning of the results, and how the approach could be applied in future work.