Evaluating IS Quality: Exploration of the Role of Expectations on Stakeholders' Evaluation

Abstract

IT Evaluation is essential, given that the value of investment in the IT industry is currently almost $2 trillion US. There is no doubt that an effective organisation will try to evaluate IT effectiveness, by linking performance measures with a financial perspective (i.e. a shareholders’ view); an internal business perspective (i.e. company planning for excellence); a customer perspective; and the innovation and learning perspective (i.e. the means to improve and create value), in order to move consistently forward. The last three perspectives are at times derived by using the same measures/instruments, via an interpretive approach based upon views of different tiers of stakeholders. Such an approach reflects a movement away from the more technical measures like benchmarking. Instead, IT effectiveness is evaluated in terms of the use of IT, or success of IT outcomes, through seeking to understand the effectiveness of the delivered IT application to the job performance of stakeholders. The merit of this interpretive approach is increasingly applicable to sectors like ecommerce, where it is very apparent that customers are concerned with the effectiveness of such IT applications. With regard to IT research, the interpretive approach was initially crystallised in the Success Model formulated by DeLone and McLean (1992). Their evaluative tools were Use and User Satisfaction. However, if research in related industries is considered, it rapidly becomes apparent that evaluation of quality is a more highly regarded approach. In seeking to adapt this approach to IT, it is important to consider the key components of an IT system, for which effectiveness would be measured in terms of quality; what quality means in an IT context; and how stakeholders internally derive an evaluation of such quality. In summary, this chapter reports on research which has produced a redefined IS Success Model, in which quality is the key to effectiveness. It also reports results of a related empirical study, which reaffirmed this IS Success Model and then investigated whether quality was better measured in terms of stakeholders’ expectations for IS performance and their perceptions of actual performance, or whether measurement of perceptions alone provided sufficient understanding of IS quality/effectiveness.