Responsibility for the Harm and Risk of Software Security Flaws

Background

One of the challenges in understanding who should ultimately be responsible for the harm and risk caused by security flaws is our lack of a full understanding of the nature of information technology risk. “As systems become more complex and interconnected, emergent behavior (i.e., unanticipated, complex behavior caused by unpredictable interactions between systems) of global systems exposes emergent vulnerabilities” (Computing Research Association, 2003, pg. 21). This complexity and emergence make risk assessment hard. Our existing mathematical/statistical risk models are based on independent failures, where “a component failure in one part of the system does not affect the failure of another similar component in another part of the system. This leads to especially beautiful and useful models of system failure that are effectively applied thousands of times a day by working engineers” (Computing Research Association, 2003, pg. 21). Unfortunately, these models are not transferrable to networked systems where failures are interdependent, not independent.

We need models that can account for dependencies between system components in a manner that sheds light on how the behaviors of system components interact to lead to system failure. Progress in interdependent risk measurement will enhance the effective management of investment. “Without an effective model, decision-makers will either over-invest in security measures that do not pay off or will under-invest and risk devastating consequences” (Computing Research Association, 2003, pg. 21). Interdependencies also pose considerable challenges when it comes to assigning liability, and formulating reasonable policy and associated compliance.