Information System success is difficult to measure directly. Because of the influence of non-controllable variables, it actually seems to be impossible to directly compute or determine the contribution of Information Systems to organizational performance, or to overall organizational effectiveness. As an alternative, perception of system success is often used as a surrogate measure. However, this raises the question of the validity of this surrogate measure. In this chapter, the authors describe a survey aiming to investigate the validity of this surrogate measure. Results show that there is reason to doubt the usefulness and validity of surrogate measures for objective system success.
TopInvestigating System Success
A notion of ‘system success’ that is widely used, was defined, by Delone and Mclean in their original 1992 paper (Delone & McLean, 1992), as ‘effectiveness’ and ‘organizational impact’. Similar definitions can be found in Doll and Tokzadeh (1998) and Ives et al. (1983). We will adopt the definition of Delone and Mclean (1992) and therefore assume that information system success indicates the degree to which the goals of the organization have been furthered. Assuming that an organization has decided on its goals, these can be translated into objective measures.
When investigating system success, direct measurement is therefore an obvious approach. However, empirical studies reported mixed findings. There is some evidence to support the idea that information systems (IS) improve the efficiency and effectiveness of a firm’s operations (e.g. Li, 1997), and that IT-related resources have indeed a potential for competitive advantage (e.g. Lüneborg & Nielsen, 2003; Mata et al., 1995). However, most research in this area found little evidence for a convincing, consistent relationship between IT investment and organizational performance (e.g. Farbey et al., 1999; Gallagher, 1974; Hitt & Brynloffson, 1996; Raymond et al., 2001; Saunders & Jones, 1992). Many publications underscore the proposition, that the benefits of IT investments are very difficult to measure (e.g. Bharadwaj et al., 1999; Davis et al., 2003; Gelderman, 1998). Because of the influence of non-controllable variables, such as changes in competitive offering or technology developments in the current product mix, it actually seems to be impossible to directly compute or determine the contribution of information systems to organizational performance or on overall organizational effectiveness (e.g. Gallagher, 1974, Kusters et al., 2002; Skok et al., 2001). As a consequence of these measurement problems, proxy measurements for IS evaluation purposes have gained considerable acceptance. In the absence of objective measures, many researchers and practitioners have turned to perceptual surrogates (McHaney et al., 2002) by focusing on perceptions of IS success.