Critical Success Factors for an Effective Security Risk Management Program

This paper evaluates the perceived effectiveness of the security risk management (SRM) programs at two Fortune 500 firms using qualitative and quantitative methods. Layers of management and staff from both firms participated in the study. Perceived effectiveness of their SRM programs was based on nine critical success factors (CSFs). Six initial critical success factors (CSFs): executive management support, organizational maturity, open communication, risk management stakeholders, team member empowerment, and holistic view of an organization were extracted from organizational role theory. They were confirmed and synthesized with three additional CSFs (security maintenance, corporate security strategy, and human resource development). A survey based on the CSFs was implemented at the two firms. Although both firms are Fortune 500 technology companies, their perceptions of current perceived SRM effectiveness differ significantly.


INTRodUCTIoN
According to an Ernst & Young Global Information Security Survey (Bandyopadhyay et al., 2009) organizations are increasingly recognizing information security risks and are improving the effectiveness of their information security programs. However, a large portion (64%) of the survey respondents indicated that the level of employee security awareness was either a significant or a considerable challenge in meeting their information security initiatives. Lack of compliance with information security policies is a major problem (Siponen & Vance, 2010). In addition, outsider threats, such as viruses and system penetration attacks continue to increase in cost and complexity.
Traditionally, IS security research has focused on its technological aspects. However, the problem has a "behavioral root" (Workman & Gathegi, 2007) and is subject to both psychological and sociological actions of people (Parker, 1981). Recent research has focused on insider threats (Sneha & Varshney, 2009). Since users interact with information systems on a regular basis in their business activities, how they use the systems and whether they follow established measures will ultimately influence the overall security of an organization's information systems.
Information security is a phenomenon that occurs in waves, progressing from technical to managerial to institutional and finally to information security governance (von Solms et al., 1994). Although methods of research in information security have been proposed and compared at length (Siponen, 2005), there exist few organizational level studies that employ theoretical rigor. Organizational systems are less secure if top managers, middle managers, and employees neglect information security procedures (Straub & Welke, 1998). Studies have shown that issues become more complex when executive management is unable to view risk from all perspectives (March & Shapira, 1987). For example, management may not consider risk takers motivated by factors other than personal incentives. They may also believe that organizations generally inhibit risk taking.
Security risk management (SRM) refers to a series of mechanisms put in place by an organization to counter or prevent information security related events (Blakley et al., 2001). Examples of such mechanisms include implementation of clearly defined information security policies and secure computing practices (Spears & Barki, 2010). An information security event may include factors such as insider threat, malware, and unauthorized access. Since SRM impacts the organization as a whole and focuses on confidentiality, integrity, and availability of data, it is imperative that effective SRM policies and practices be established and followed.
The overall objective of SRM is to enable an organization to handle information and data adequately. As such, data and information should be safe from potential threats. SRM is not a standalone activity. Instead it should be an integral part of the processes throughout an organization (Dhillon, 2007). This includes addressing potential threats, educating personnel in security awareness, and establishing and executing security policies. Considering the overarching impact of an SRM program, it is surprising to note that little research has been conducted in this area. Kotulic (2001) developed an instrument that provided a starting point for the development of theory-based guidelines to manage the SRM process. His model included a direct relation between executive management support and SRM program effectiveness. However, he was unable to test his theoretical model. Although he had confirmed focused interviews with five firms, all firms declined when they saw the preliminary questionnaire. Over a course of several months, he contacted 38 additional firms. Finally, one firm agreed to participate in the study, but on a limited basis. Based on interview results, the survey questionnaire was modified and sent to over 1,500 top management team members of large (greater than 500 employees) firms. Unfortunately, less than 100 surveys were returned, resulting in a response rate of less than 2%. This was attributed to the sensitive nature and complexity of the survey questions (Kotulic & Clark, 2004).
In order to be effective, security controls must be in line with the goals and objectives of an organization (Khansa & Liginlal, 2011;Spears & Barki, 2010). Therefore, it is important to focus on the information needed to attain these goals and objectives. Critical success factors (CSF) are "things" that must go well to ensure success for a manager or an organization (Rockart, 1979). We purport that employee (management or staff) understanding of CSFs as they relate to SRM can allow an organization to maximize the overall effectiveness of its SRM program. Therefore, we expanded upon Kotulic's research by incorporating a modified version of the CSF method. Initial CSFs were extracted from Role Theory, which provides insight into recurring patterns of actions that are considered important for effective role functioning in an organization (Kahn et al., 1964). We believe this was necessary to not only provide theoretical rigor, but also assist in easing some of the concerns the organizations may have had about an intrusive topic such as information security. Through a series of interviews, we extracted additional CSFs, which formed the basis of a synthesized list of CSFs.
The purpose of this study was twofold: 1) identify CSFs as they relate to SRM, as perceived by both management and staff, and 2) ascertain how management and staff perceive the effectiveness of their SRM policies and procedures in attaining the goals associated with these CSFs. Currently, no known studies have concentrated on the link between management and staff in terms of establishing and maintaining effective SRM policies. Executive management may have varying perspectives of expected SRM strategies than those of the staff. This can impact the actual effectiveness of SRM strategies. Therefore, the primary research question addressed in this study is: What is the CSF impact on the perceived effectiveness of an organization's SRM program?
The remainder of this paper is organized as follows. First, we provide a literature review, focusing on CSFs and SRM. Next, we describe our mixed method research design, along with detailed discussion of both qualitative and quantitative portions of the study. This is followed by detailed discussion of the results and contributions to practice. We conclude the paper by discussing limitations and suggestions for future research.

LITERATURE REVIEw
This section first focuses on studies that investigated the critical success factors concept, followed by studies pertaining to organizational information security.

Critical Success Factors (CSF)
The CSF method was initially proposed (Rockart, 1979) to help CEOs specify their information needs related to critical firm issues so that systems could be developed to meet those needs. CSFs are intended performance consequences of systems and behaviors within the firm, which are strongly related to the achievement of desired firm objectives. Benefits of CSFs include 1) identifying factors for management scrutiny developing, 2) establishing measures for evaluation, 3) focusing attention on significant data to be collected, 4) accommodating change within an organization, and 5) assisting in the planning process (Rockart, 1979).
In IS, the CSF method has been introduced as a mechanism for aligning IT planning with the strategic direction of an organization (Rockart, 1979). User acceptance is a major benefit of using the CSF method. Managers seem to intuitively understand the thrust of the CSF method and endorse its usage as a means of identifying areas of concern in an organization (Boynton & Zmud, 1984). CSFs can also be used as an MIS planning tool by interviewing multiple levels of managers in an organization (Bullen & Rockart, 1981).
Organizational level studies that consider SRM in the context of an actual business setting are currently lacking in IS research (Weiser, 1991). This might be due to the fact that security is considered to be intrusive in nature (Kotulic & Clark, 2004). The following section provides a review of information systems security policies proposed and their use in the organizations.

RESEARCH METHod
We employed a mixed method research design approach that incorporated a combination of data collection and analysis methods on separate samples to examine user perception of SRM effectiveness. Data were collected at two firms, Companies A and B. Both are multinational Fortune 500 technology firms with established security risk management programs. We concentrated on a single location for each firm. Each location was within the United States, but in different parts of the country. Participants belonged to different layers of management (i.e., executive management, middle management, and lower management) and staff. We only considered full-time employees. In both companies, there were a greater number of staff participants, compared with management. However, this is not considered a limitation of the study, since an effective organization will typically have a pyramid structure (Sennewald, 2003). Qualitative (using interviews) and quantitative data (using a survey) were collected at Company A. Management and staff at Company B responded to the same survey that was used for Company A.
Various researchers have addressed methodological issues such as lack of control and generalizability that arise when a case study is conducted (Datta, 1982;Dukes, 1965;Huberman & Crandall, 1982;Miles, 1982). To counter these issues, guidelines have been presented for the positivist case research paradigm (Lee, 1989;Yin, 1994). These guidelines have also been successfully applied (Sarker & Lee, 2003) and are summarized in Table 1. This section explains techniques that have been shown to enhance research rigor and how they were implemented in this study. We also enhanced the guidelines, as indicated in the table.
Since information security is considered a naturally intrusive topic, extensive informal conversations between the researcher and some of the management at both companies (Company A and Company B) had already taken place prior to execution of the study. The CSF method was selected due to its nature of relying on dialogue, hence easing management concerns about the topic. This allowed the development of a reasonably comfortable relationship between employees of both companies and the researcher, which is an essential component of security based research (Kotulic & Clark, 2004). Prior communication also allowed Company A to grant us permission to study the in-depth aspects of their SRM implementation through interviews and questionnaires, which formed the core component of a CSF-based case study. On the other hand, Company B's employees were asked to complete the same survey that was administered at Company A. This allowed us to compare quantitative results at both companies.

Qualitative data Collection at Company A via CSF Method
There is no standard procedure for CSF data collection and analysis (Bergeron & Begin, 1989). Rockart (1979) suggested that CSFs should be collected during three to six hours of interviews with the CEO, but his concept only focused on the CEO's information requirements. As the problem and organizational scope of CSFs has broadened, consultants and researchers have used alternative methods such as "onion technique" interviews and analysis of interrelated organizational activities (Dickinson et al., 1985), an a priori list of CSFs from literature and a mailed questionnaire (Sabherwal & Kirs, 1994), and most importantly interviews followed by questionnaires to implement CSFs (Guynes & Vanecek, 1996).
To confirm/disconfirm the initial CSFs, structured and unstructured interviews were conducted with key personnel across all layers of management and staff. In all, 32 employees at Company A took part in this portion of the study. These employees were randomly selected, and notified in advance about the option of interviewing. The breakdown of employees by level of employment is presented in Table 2.  Lee (1989) and Yin (1994) Criterion

Guidelines How the Guidelines Were Followed in this Study
Internal validity Pattern matching "Natural Controls" were used whenever possible.
Interviews completed within approximately one month to prevent maturation effect.
As part of the CSF confirmation/disconfirmation process, each participant was asked to create hypothetical scenarios (vignettes) related to potential violations of an SRM policy and their impact on Company A. Unlike the standard use of researcher-provided scenarios (Wason et al., 2002), we asked the participants to create their own scenarios, based on situations that were mentioned in Company A's SRM policy. The use of vignettes is recommended as a way of relating to sensitive survey questions (Lee, 1993). We contend that presenting a respondent with an opportunity to create a scenario rather than asking direct questions about their organization's SRM policies resulted in a more honest gauge of their perceptions regarding information security.
As shown in Table 2, 14 of the 32 interviewees provided scenarios. Examples of the scenarios are presented in Appendix B. From these scenarios, we were able to glean quite a few points. Management scenarios were more concrete and highlighted a more strategic level of thinking and greater understanding of the SRM process. For example, an executive manager's scenario (see Appendix B Scenario 1) clearly specified the process in the event an employee deleted a customer's file. Conversely, staff scenarios (for example, refer to Appendix B Scenario 2) had a much narrower scope, focusing more on an immediate advantage in a position of authority as opposed to the consequences of an action.

Confirmation of Initial CSFs and Extraction of New CSFs
As each interview progressed there was clearly a difference in opinion between management and staff with regard to the most important components of an effective SRM program. Senior members of the SRM management team were acutely aware of the challenges Company A faced in developing an SRM program that complied with both U.S. and European regulations. Since it is a global organization, this was one of the critical requirements for Company A. According to one executive, Company A dealt with a patchwork of "disparate and over-lapping state and federal regulations, along with privacy rules laid out by individual corporate partners." Within the European Union, it dealt with "the data protection directive, which unlike U.S. regulations such as HIPAA or Sarbanes-Oxley acts, provides few specifics as to how these privacy requirements should be met." While creating the SRM program, management therefore focused on the need to establish a consistent set of requirements common to various U.S. and EU jurisdictions, while keeping in mind Company A's own standards for protecting customer and supplier data. This was through enhanced security features such as encryption. This was also why, according to a middle manager, Company A focused on creating in-house security tools as part of their corporate security strategy. It allowed the organization to build a foundation that was both deep and broad, rather than a series of narrow solutions that addressed regulations on a case-by-case basis.
Overall, staff knew very little about the various U.S. and EU directives. However, most members of staff agreed that it was important to encourage growth of corporate security strategies because it made the organization proactive instead of reactive. This, according to a staff member, also "prevented waste of staffing and budget resources." The same person elaborated on how the current method of assessing the SRM program also had its disadvantages. Accordingly, when a division was informed that an assessment would occur, it changed its current practices to what was required as part of the SRM program. However, as soon as the division passed the assessment, the makeshift (required) processes were removed in favor of the initial practices. Although most staff members were not familiar with the plethora of multi-national compliance requirements Company A was required to follow, they were well versed in other security services such as access control, encryption, employee training, and policy updates. According to a staff employee, Company A "is not immune to threats such as cyber-theft, and cyber-espionage by hackers, malware, and malicious insiders. It now uses logs for forensic analysis, and has detailed access control procedures, to try to prevent all types of cyber security incidents." Both management and staff concurred on the importance of updating policies. According to management, business divisions usually initiated risk assessments based on each division's prior results. Division heads were primarily responsible for ensuring that recommendations were implemented and that periodic updates were conducted and verified.
Discussions during these interviews confirmed the significance of the 6 initial CSFs and resulted in three additional CSFs: security maintenance, corporate security strategy, and human resource development. In subsequent interviews, the personnel were presented a final list of CSFs, and these CSFs were agreed upon by both management and staff. A point to note is that we obtained no new information regarding CSFs after interviewing 20 participants. Hence, we concluded that interviewing 32 participants for this phase of the study was sufficient. The CSFs and what they entailed are described next.

Executive Management Support
During the interviews, the role of executive management support as a CSF with respect to the SRM implementation became clear. Management routinely carried out risk assessments of each business division. Staff considered this relevant to SRM effectiveness because it gave everyone the impression that risk assessments should be taken seriously at all organizational levels.

Organizational Maturity
Before carrying out a risk assessment in a business division, the security management team conducted an audit of the way things were presently being done. According to senior staff members, this could shed light on some security protocols that might have been "forgotten with time." Examples included authorization and authentication processes, disaster recovery, physical security, and intrusion detection and incident response. A focus on current methods with respect to existing policies forms a core component of organization maturity.

Open Communication
Company A's security assessment team used tables, questionnaires, and standard report forms to facilitate the functioning of its SRM program. Employees who participated in the risk assessments were familiar with these tools, which facilitated effective communication of results of the various risk assessments. The employees also stated that the extensive use of this simple method of open communication increased understanding of SRM objectives between management, system support staff, and security specialists.

Risk Management Stakeholders
It is interesting to note that Company A relied exclusively on in-house personnel instead of an outside contractor to carry out risk assessments of their SRM program. According to a lower level manager, this was necessitated by the nature of their work and the risks involved in the event the contractor leaked sensitive information about some of the company's programs.
Employees selected to carry out the risk assessments were well aware of the entire SRM program. This is in line with the description of what constitutes the definition of risk management stakeholders.
According to management, use of in-house employees for this purpose better enabled them to explore a greater variety of risks.

Team Member Empowerment
Each business division was empowered to request a test of the SRM program via an assessment. The responsibility of following up on resulting recommendations also lied primarily with the requesting division. According to an executive manager, each business division was best qualified to determine when an assessment was required and to ensure that recommendations for risk reduction techniques resulting from the assessment were implemented effectively.

Holistic View of an Organization
Groups and individuals were designated as focal points to oversee the various risk assessment processes. Due to the overarching nature of Company A's SRM implementation, it was necessary that these groups and individuals had a clear view of the effectiveness of the SRM program. According to management, this facilitated the performance, planning, and reporting associated with Company A's SRM program, and ensured that enterprise wide issues were appropriately addressed.

Security Maintenance
Security maintenance is defined as a set of controls and best practices, such as policy updates, access control, and physical and personnel security that organizations should adopt to maintain a sufficient security standard (Dhillon, 2007). This CSF entails many of the security features that form the backbone of Company A's SRM program. Features include role-based access control, encryption standards, physical security, and policy updates. An interesting feature of this control is that if employees wish to view any of the security bulletins they can only access information that may pertain to them in their current capacity. According to one of the participants interviewed, this allowed for "removal of unnecessary clutter, and motivates most to read the policy [in the first place]." The primary focus on encryption standards pertained to company laptops that most employees took home with them. Each laptop had full disk encryption that used Trusted Platform Module (TPM) technology. This was implemented after a security incident involving a lost laptop. Company A's site had a high degree of physical security. Visitors were not allowed to enter the premises without an escort. They had to first pass through a scanner. Next, they were physically searched by a guard. Company A employees only had to pass through the scanner but were randomly asked to volunteer for a physical search. Also, all lobbies, corridors, and common areas such as the cafeteria had closed circuit television monitoring.
Both management and staff as a major area of concern repeatedly mentioned policy updates. Although they agreed that this was a CSF, 19 of the 32 employees interviewed expressed concerns about the lack of security bulletin updates. Some of the bulletins had not been updated in almost six years. According to one employee, the lack of updated compliance guidelines could pose threats such as "covert downloads of a Trojan, malicious employee attacks, use of insecure cloud computing applications, and botnets."

Corporate Security Strategy
Corporate security strategy is defined as steps such as development of technologies undertaken by management to incorporate security needs as a fundamental function of the corporation (Dhillon, 2007). As previously stated, management focused on bridging the gap between U.S. and EU regulatory compliances to assure that all regulations and requirements were satisfied. These regulations, in addition to Company A's own security standards, called for establishment of tailor-made procedures and protocols that integrated the disparate requirements. Company A encouraged the creation of cutting-edge technologies and considered it as one of the cornerstones of its SRM program. Staff members agreed with the importance of in-house security software and its role in providing an effective SRM program.

Human Resource Development
Human resource development is defined as the existence of a company framework that focuses on development of personal and organizational skills, knowledge, and abilities. With respect to SRM at Company A, this could include opportunities such as employee risk and security training. This could be quantified through focus on various security certifications such as Certified Information Systems Security Professional (CISSP) and Certified Information Systems Auditor (CISA) (Dhillon, 2007). Staff members expressed an interest in knowing the background knowledge and experience of personnel who dealt with security related issues. The reason for their concern pertained to privacy and confidentially. Management stated that although "extensive logical and technical controls exist[ed], personnel experience was just as important as the other aspects." They felt that in order to reduce human error, fraud, or misuse of company property, those in charge of the process had to be carefully screened and mandated to go through education and training.

Ranking of CSFs
We asked the participants to rank the CSFs according to their perceived importance using Q-sort. This technique has been recommended in previous studies (Stephenson, 1953). As part of this method each employee was given a set of cards, each of which had a CSF. They were then asked to arrange the cards in order of perceived significance (from highest to lowest). The ranking provided us with more information with regard to employee preferences based on their own subjective scale. Results are presented in Table 3 in order of decreasing importance of each management level and staff. A total of 40 employee participated in the Q-sort exercise, which included all 32 or the interviewees and 8 more subjects who participated in the survey.
It is interesting to note that executive management support is considered the most important CSF by all management layers (executive, middle, and lower), whereas staff considers open communication to be the most important. Most of the CSFs in each column are fairly close to each other in terms of rank. For example, team member empowerment appears toward the lower half of each of the rankings.

Quantitative Results for Company A
In order to gauge differences in management and staff's perception of the effectiveness of SRM policies and practices, we administered a multi-item questionnaire, based on the synthesized list of CSFs, to both management and staff. Participation was voluntary. Participants were given the option of completing either an electronic or paper based survey. A portion of the questionnaire used to measure the CSFs was pre-validated by Kotulic (2001). Kotulic measured perceived effectiveness of an SRM program using a questionnaire that was structured for a case study. Hence, it was appropriate for our study. Each construct had multiple items associated with it in the questionnaire, each representing a CSF. Perception of SRM effectiveness was also a part of the questionnaire. As a means of minimizing response bias caused by boredom or fatigue, no more than five items per construct were presented (Schriesheim & Eisenbach, 1995). However since not all CSF constructs were represented in Kotulic's initial survey, we added new items prior to validation and reliability testing. Appendix A presents the entire questionnaire.

Validity and Reliability
We asked 7 professors and 11 doctoral students at a large North-Eastern university to review the updated questionnaire. We also asked a security manager at Company A to ascertain if the questions were appropriate for each construct. This assisted in enhancing the construct validity of the questionnaire, as specified by Nunnally and Bernstein (1994). We made minor refinements to the survey and administered it to 135 employees of Company A to test for reliability and construct validity.
We used Confirmatory Factor Analysis to gauge construct validity. The results of the crossloadings are presented in Table 4. The constructs are coded as follows: SRM effectiveness (SRM), executive management support (EMS), organization maturity (OM), open communication (OC), risk management stakeholders (RMS), team member empowerment (TME), holistic view of organization (HVO), security maintenance (SM), corporate security strategy (CSS), and human resource development (HRD).
As shown, the factor loadings are all above the suggested threshold of 0.6 (Chin, 1998). In addition, items that measure the same construct have higher loadings than those measuring other constructs. This suggests acceptable convergent and discriminant validity. To further assess construct validity, Table 5 presents the correlation between the constructs, with the diagonal elements being the square root of the average variance extracted (AVE). The AVE of each construct exceeded 0.5, the benchmark for convergent validity (Fornell & Larcker, 1981). In addition, the square root of the AVE of each construct was greater than the correlation between the construct and other constructs, suggesting adequate discriminant validity.
The reliability was assessed using Cronbach's alpha and composite reliability. Table 6 shows that the alpha value and composite reliability for each construct was above 0.7, the suggested threshold for adequate reliability ().
Once we ascertained the reliability and validity, we administered the survey (either electronic or paper version) to the rest of the employees at Company A. The next section provides the quantitative results from the participants of the study.

RESULTS
A total of 272 out of 378 employees, excluding those who participated in the pilot study, took part in the survey portion of the study at Company A. Hence, the response rate was 71.96%. Table 7 provides statistics on the total number of participants, along with demographic information.
We compared the management layers (executive, middle, and lower) with the staff to measure differences in perception in the effectiveness of current SRM policies and practices. In each case, we established linear regression models with dummy variables with interaction effects. The independent variables are the nine CSFs along with dummy regressors for management and staff. The dependent variable is SRM effectiveness. Using the data, we carried out different levels of data analyses with regard to one of the purposes of this study: identify CSFs that may highlight differences in perceptions of SRM effectiveness among management and staff.
We checked and corrected for violation of any of the OLS assumptions. We also checked for heteroskedasticity using the Breusch-Pagan (BP) test (1979). If the BP test was significant, indicating inefficient OLS, we used Generalized Least Squares (GLS) estimators.
Considering that there were a limited number of executive and middle managers, it was more prudent to incorporate one interaction effect per layer and compare two groups at a time. By concentrating on the interaction effect, we were able to gauge the perceptions of employees at different levels on each CSF and its impact on SRM effectiveness. This resulted in 54 multiple regression equations. A general form of the regression equation used was: SRM = β 0 +β 1 EMS +β 2 OM+ β 3 OC+ β 4 RMS+ β 5 TME+β 6 HVO+β 7 SM+β 8 CSS+β 9 HRD + δ 1 Layer + X 1 CSF*LAYER +ε (1)

Executive Management (EM) and Middle Management (MM)
To gauge the differences in perception between executive management and middle management with regard to how each CSF influences current perceived SRM effectiveness, we used equation (1). EM was "0" in the event a person belonged to middle management and "1" if the employee belonged to executive management. Since the focus was on the interaction term, the tabulated results that follow only provide the relevant regression coefficients. Table 8 provides results when considering each CSF as an interaction term.
The results presented in the proceeding tables refer to a different regression for each CSF. Since we compared two groups for each regression, the critical value was the interaction between the CSF and the binary variable for the group.
As shown, executive and middle management differ significantly in terms of perceived effectiveness of SRM policies and procedures, as related to all nine of the CSFs. Overall, executive management has more positive perceptions of the effectiveness of policies and practices related to executive management support (EMS), organizational maturity (OM), open communication (OC), security maintenance (SM), and human resource development (HRD). Conversely, they had less positive perceptions of the effectiveness of policies and practices related to risk management stakeholders (RMS), team member empowerment (TME), holistic view of the organization (HVO), and corporate security strategy (CSS).

Executive Management (EM) and Lower Management (LM)
None of the CSFs was significant (Table 9). Hence, there was no significant difference in perception between executive and lower management with regard to the CSFs pertaining to SRM effectiveness.

Executive Management (EM) and Staff (ST)
Although there was no significant difference in perceptions of SRM effectiveness between executive and lower management (Table 9), executive management and staff differed for each CSF (Table 10). In each instance, executive management has a less positive view than staff regarding the effectiveness of the current SRM policies and procedures, as related to the CSFs although it is not statistically significant.

Middle Management (MM) and Lower Management (LM)
Table 11 shows results for the model when the interaction terms for each CSF are included. The results were significant for the EMS, OC, TME, HVO and SM interactions. Overall, middle management perceives the effectiveness of SRM policies and practices related to TME and HVO more positively than does lower management. Conversely, they have less positive perceptions of the effectiveness of SRM policies and practices related to EMS, OC, and SM.

Middle Management (MM) and Staff (ST)
Table 12 compares interaction terms between middle management and staff. As shown, middle management has less positive perceptions than staff in reference to the effectiveness of SRM policies and practices related to EMS, RMS, SM, CSS, and HRD.

Lower Management (LM) and Staff (ST)
Table 13 compares interaction terms between lower management and staff. As shown, there are significant differences in seven of the nine CSFs. Lower management has a more positive perception of the effectiveness of EMS, but has less positive perceptions of the effectiveness of OM, OC, HVO, SM, CSS, and HRD.

Quantitative Results for Company B
As previously stated, both Company A and Company B are Fortune 500 technology companies. As such, Company B was selected to determine if both firms agreed in reference to the CSFs and their impact on SRM effectiveness. We administered the same questionnaire to full time employees at one of the locations of Company B. One hundred fifteen (115) out of 132 personnel at Company B responded to the survey. Hence, the response rate was 87.12%. Table 14 presents participant demographics. To analyze the survey results, we used the same regression modeling technique employed for Company A. The following sections present the perception of SRM effectiveness at Company B. We included Company A results for each regression analysis for comparison.

Executive Management (EM) and Middle Management (MM)
Significant interactions existed for EMS, OM, OC, RMS, TME, and HVO (Table 15). Company B's executive management is less positive than middle management in reference to HVO. All other significant interactions indicated that their perceptions toward SRM effectiveness related to these CSFs were positive.

Executive Management (EM) and Lower Management (LM)
Table 16 compares interaction terms between executive and lower management. As shown, executive management has more positive perceptions regarding the effectiveness of their SRM policies and practices related to EMS, OM, and HRD. All other interactions were not significant. As shown, there were no significant differences between Company A's executive and lower management.

Executive Management (EM) and Staff (ST)
Table 17 compares interaction terms between executive management and staff. Interestingly, these results are very different from those of Company A. Whereas Company A's executive management had less positive perceptions regarding all nine CSFs, Company B's executive management was either more positive (EMS, OM, OC, SM, and CSS), or results were not significant.

Middle Management (MM) and Lower Management (LM)
When compared to lower management (Table 18), middle management considers EMS, OM, OC, CSS, and HRD to have a more positive impact on SRM effectiveness. Once again, this is in contrast to Company A.

Middle Management (MM) and Staff (ST)
Table 19 shows that all interactions between middle management and staff were significant. For each CSF, middle management has a more positive perception of the effectiveness of SRM policies  and practices. These results are quite different from those of Company A. Although not statistically significant in all instances, Company A's middle management considers each of the CSFs to have a less positive impact on SRM effectiveness when compared to staff.

Lower Management (LM) and Staff (ST)
As shown in Table 20, when comparing lower management to staff at Company B, all CSFs were significant and positive. Once again, these results differ from those of Company A. Overall, Company A's lower management is less positive than staff in reference to the CSF impact on SRM effectiveness.
Where, the dummy value MGT was "1" if an employee was a part of any of the management layers, and "0" otherwise (staff). Table 21 shows the results for both companies Unlike the previous regression models in which the focus was on individual interactions between a CSF and a group, results from equation 2 have to be interpreted as a whole. Hence its macro-level description quality. For Company A, the negative coefficients for RMS, TME, and SM, and MGT imply that neither management nor staff is satisfied with the current policies and practices with respect to the three CSFs. Conversely, at Company B, management and staff have a positive perception about all CSFs and their impact on SRM effectiveness. Table 21 shows that at Companies A and B, each group considered each of the CSFs important for SRM effectiveness. However, they varied on the degree the CSFs impacted current perceived SRM effectiveness. This mirrored results from the previous regression models.
From the interviews with employees of Company A, we found additional information related to risk management stakeholders, team member empowerment, and security maintenance.

RESEARCH CoNTRIBUTIoN
One of the main contributions of this research is that we were able to obtain sensitive information related to information security and risk management. In addition this is one of few studies on information security that involve Fortune 500 companies. This research confirmed six initial CSFs (executive management support, organization maturity, open communication, risk management stakeholders, team member empowerment, and holistic view of organization) and discovered three new CSFs (security maintenance, corporate security strategy, and human resource development). Each of these CSFs had an underlying theme -a congruence of SRM program objectives and policies that would make them effective. The use of a widely-recognized theory from organizational studies in a case study environment allowed greater focus on the roles that levels of management and staff play with regard to SRM effectiveness. We used a combination of qualitative and quantitative methods to validate the finding. The impact of roles was quantified through multiple regression models with and without interaction effects. This study also contributes to the CSF literature by applying the CSF method across not only layers of management, as has been done in previous studies, but also the staff layer. SRM implementations in organizations are complex and enterprise-wide. Therefore, for an effective SRM program, all stakeholders should be considered.

IMPLICATIoNS FoR ACAdEMICIANS ANd PRACTICE
For academicians, this study shows that the CSF method can be modified to address complex organizational issues such as SRM effectiveness. As stated throughout this paper, information security research at the organizational level has constantly faced challenges due to its intrusive nature. In most cases, management approval is needed to carry out research of such magnitude. The CSF method is a management-friendly form of research. It allows for the use of structured and unstructured dialogue with the added opportunity of the researcher communicating CSFs back to management for approval. This method, if explained properly to management, has the potential to deter many of the fears associated with information security related research.
Recognizing that differences in perception exist among layers of management and staff in terms of SRM effectiveness is a vital issue that needs to be addressed by the security management team. In Company A, through results from the q-sort exercise and multiple regression models, we found that certain CSFs such as executive management support ranked consistently higher than the rest. Staff considered executive management to be the prime mover behind SRM effectiveness. Based on the observations, it is obvious that the current SRM implementation at Company A does have executive buy-in. However, there is a need to better communicate management's proactive role in SRM implementation to staff. Better communication channels will create synergy between different employee levels and encourage an effectiveness of the SRM program. We can link this objective to open communication and holistic view of the organization, two highly-ranked CSFs according to our results. For management to effectively communicate its SRM vision, the organization must have a network structure with clearly demarcated boundaries.
The absence of policy updates was one of the more glaring deficiencies pertaining to the current SRM implementation at Company A. Security by its nature is an evolving field. Therefore, it is important to ensure that security policies are as up-to-date as possible.
There are other potential implications as well. Inconsistent perception of CSFs on SRM effectiveness may cause liability issues. For example, problems associated with intellectual property can result in class action lawsuits as well as criminal liability. We found significant differences in perception among management and staff in reference to intellectual property.
The scenarios showed how a lack of understanding of a SRM program could deter employees from being as productive in their work as they would normally be. This loss of productivity could prove to be critical for an organization since it could result in lost revenue and/or customers.

LIMITATIoNS
One of the potential limitations of this study is that it considers CSFs in the context of only two organizations. Therefore, results cannot be generalized across other companies. This limitation is somehow reduced by the fact that both companies are multinational Fortune 500 technology firms with an established security risk management program. However, due to its sensitive, complex nature, this was meant to be an exploratory study. Advances in both the science and practice of information systems and allied disciplines (e.g. management, computer science, and psychology) hinge on results derived from exploratory research. As part of role theory development, Katz and Kahn (1978) initially had more than a dozen factors derived from observed phenomena as part of exploratory research. They made it a life's work of testing those factors in different environments before settling on a core list of role theory tenets applicable to organizations. A similar path can be followed by testing the synthesized list of the CSFs found in this study across organizations in different industries.
Another potential limitation may be that the research method, albeit management-friendly, may have biased employees toward a particular CSF. Care was taken with regard to this problem via the use of standardized scripts and an overall sense of neutrality and passive role on the part of the researcher.
Finally, the issue of adequate sample size needs to be discussed, especially with regard to the total number of participants in Company B. At the estimation stage, the problem of sample size is largely removed by the use of unbiased estimators. Under random sampling, the expected value of the unbiased sample estimator will be the true parameter value, regardless of the sample size. However, as in any statistical estimation, the statistical consistency of the estimator is improved as the sample size increases. A small sample size is also not a concern in exact cases of inference where sampling distributions are either independent of the sample size or dependent upon the degree of freedom, thus explicitly incorporate the sample size. However, a small sample size could be of concern in cases of inference that rely on asymptotic sampling distributions. The regression procedures employed in this study are consistent with the small sample properties.

SUGGESTIoNS FoR FUTURE RESEARCH
This exploratory work has set the stage for future research in the area of organizational information security. Future studies can include more companies of various sizes from different industries, as well as extracting additional CSFs. As shown in Company A, there were significant differences in perceptions of the effectiveness of the SRM program. Further studies should explore why these differences exist, and how problems can be resolved.

CoNCLUSIoN
This study is the first to provide empirical evidence of the relation between CSFs and SRM effectiveness based on the perception of management and staff of two Fortune 500 companies. The study examined the differences between perceptions of management layers and staff, and found significant differences between groups based on data analysis using multiple regression models. Information security is an ongoing effort that requires regular updating on the policies as the business and regulatory requirements evolve. The qualitative part of the study found that more effective implementation of SRM can be achieved through fostering enhanced communication between layers of management and staff for greater perception alignment.

open Communication (oC)
12) When a formal security policy initiative is launched, visibility is given to the event through devices such as management presentations and question/answer forums.
Not at all To a large extent 1 2 3 4 5 6 7 13) Management communicates visibly and seriously regarding the need to protect the confidentiality of sensitive information.
Not at all To a large extent 1 2 3 4 5 6 7

Risk Management Stakeholders (RMS)
14) The current security policy is the result of inputs from many members of our organization. Not at all To a large extent 1 2 3 4 5 6 7 15) Auditors and security personnel are involved in design changes in information systems. Not at all To a large extent 1 2 3 4 5 6 7 Team Member Empowerment (TME) 16) Getting authorization to access data that would be useful in my function is time consuming and difficult. Not at all To a large extent 1 2 3 4 5 6 7 17) Data that would be useful to my function is unavailable because we do not have the right authorization.
Not at all To a large extent 1 2 3 4 5 6 7 18) The decentralized organization of the unit's Information Security Services with respect to personnel who carry out security policy related work is beneficial.
Not at all To a large extent 1 2 3 4 5 6 7 19) The decentralized organization of the unit's Information Security Services with respect to securing hardware and software is beneficial.
Not at all To a large extent 1 2 3 4 5 6 7

Holistic View of an organization (HVo)
20) The organization's business objectives and goals include compliance with a broad-level security policy. Not at all To a large extent 1 2 3 4 5 6 7 21) There is strong insistence on a uniform managerial style throughout the organization. Not at all To a large extent 1 2 3 4 5 6 7 Security Maintenance (SM) 22) The role based access control procedures offered are sufficient. Not at all To a large extent 1 2 3 4 5 6 7 23) The organization takes adequate steps in updating the SRM policy. Not at all To a large extent 1 2 3 4 5 6 7

Corporate Security Strategy (CSS)
24) The organization provides adequate support for the intellectual property rights issues associated with in-house security solutions (e.g. patent support etc). Not at all To a large extent 1 2 3 4 5 6 7 25) The organization supports development of in-house security software. Not at all To a large extent 1 2 3 4 5 6 7 Human Resource development (HRd) 26) The organization offers sufficient security training to members who are directly involved with the security risk management process. Not at all To a large extent 1 2 3 4 5 6 7 27) Personnel responsible for executing the security risk management process have sufficient experience to deal with security related incidents.
Not at all To a large extent 1 2 3 4 5 6 7 APPENdIx B

Scenario 1 (Executive manager)
Jeffrey accidentally removes a customer's file from the main database. Upon realizing his error he reports the instance to his department head, who is obligated to report it to the VP of Internal Audit. Jeffrey's privileges are taken away until an investigation is carried out and completed.

Scenario 2 (Staff employee)
As a help desk employee, John has certain access to hardware that might allow him to compromise hardware or software. Also, John, due to his ability to purchase technologies below a certain threshold, may unintentionally purchase something that is inherently insecure.