A Multinational Study of IS User Satisfaction and User Performance in the Intelligence Community

Introduction

In order to effectively combat terrorism, the intelligence community (IC) must have the capability to share intelligence globally. Since the September 11 attacks, there has been a new emphasis on sharing intelligence not only between agencies but also between nations. To ensure that vital intelligence is available to all its stakeholders, the IC spends a significant portion of its budget on building information systems (ISes) every year. Confirming that these funds are being used effectively requires systematically measuring user satisfaction and user performance and providing feedback for intelligence systems. However, hardly any studies are done in this area.

In the past, various measurements have been developed to evaluate the success of ISes. It is important for the IC to understand whether these measures are still valid in this domain. In behavioral research, most of these measurements are based on user satisfaction (Melone, 1990). However, these studies did not pay enough attention to another vital measure of IS success: user performance. One possible reason is that researchers do not feel the need to measure user performance separately because of its strong correlation with user satisfaction (Etezadi-Amoli & Farhoomand, 1996; Gatian, 1994; Gelderman, 1997). Furthermore, most previous studies used survey results to measure user performance. It was reported that self-rated performance measures may be very subjective and overrated (Gelderman, 1997). This suggests that the current methods of measuring user performance using self-reported ratings could be greatly affected by user satisfaction. Despite these problems, self-rated performance measures are still widely used due to the difficulties in obtaining objective and accurate performance measures in organizational settings. In certain domains such as information retrieval (IR), where objective user performance can easily be measured by precision and recall, it was found that user performance has little connection to user satisfaction (Griffiths et al., 2007). Thus, it remains unknown whether the positive relationship between user satisfaction and user performance holds when objective measures of user performance are used. Furthermore, it is very important to identify the influence of individual differences on IS success in the IC domain. Many different user attributes such as age, gender, user experience, training, and position have been proposed as key variables determining IS success in past studies (DeLone & McLean, 1992; Sabherwal et al., 2006; Simmers & Anandarajan, 2001; Wani et al., 2017; Zmud, 1979).

Another highlight of this study is its focus on the IC domain. There are very few studies on IS success in this domain due to the difficulties in approaching users in intelligence agencies across nations. The majority of the research conducted in this area has taken place in for-profit businesses or in the private sector (Rainey et al., 1976). Previous research has shown that there are significant differences between the government-owned public sector and the for-profit private sector (Rocheleau, 2006). In addition to the lack of focus on the public sector, most of the previous studies were conducted in a single country. The management literature suggests that the differences in work and management style among employees increase when members of organizations are from diverse cultural backgrounds (Hofstede et al., 1990). Therefore, this study also fills a gap in the research by examining the information sharing and collaboration practices in the IC in a multinational setting.