Process-Based Evaluation of Hospital Information Systems: Application of an Information System Success Model (PRISE) in the Healthcare Domain

Introduction

Health care organizations and patients can benefit greatly from the appropriate and effective use of information systems (hereafter IS) (Kaushal, Barker and Bates, 2001). Effective implementation of IT in health care has the potential to save billions of dollars while reducing morbidity and mortality (Chaudhry, Wang, Wu, Maglione, Mojica, Roth, Morton, and Shekelle, 2006). As Burke and Menachemi (2004) stated in the paper, in the recent years, health care investment in IT has experienced a significant increase, which in parallel results with an increase in IT capabilities. Empirical research has shown that effective information systems in healthcare organizations are associated with reduced costs and with better care and organizational performance (Chang and King, 2005; Li and Ye, 1999), which suggests a connection between IS process improvement and organizational effectiveness. However the impact of IS implementations are hard to measure and identify (Jayasuriya, 1997). Researchers also suggest that the impact of IS is usually accidental rather than planned (Borum and Christiansen, 2006).

While the effectiveness of information systems is recognized as an important issue, the definition of IS effectiveness in the information systems literature is not yet mature and consistent (Ozkan, Hackney and Bilgen, 2007). Various models such as Capability Maturity Model (CMM/CMMI) (SEI, 2007), Control Objectives for Information Technology (COBIT) (ISACA, 2007) and Information Technology Infrastructure Library (ITIL) (OGC, 2007), have been proposed and used by the IT industry (Ozkan et al, 2007). IS management and evaluation frameworks are not limited to the above mentioned and the list includes approaches that range from the generic ISO 9001 (Braa and Ogrim, 1995), Six Sigma (Schroeder, 2007), and EFQM (EFQM, 2006; Donahue and Vanostenberg, 2000) models through IT specific models such as SPICE (SEI, 2007), ISO 9126, ISO 20000 (ISO,2007), to healthcare-IT specific models such as Q-Rec (Q-Rec,2007), Joint Commission International Information Management Standard (Donahue and Vanostenberg, 2000) and HIS-Monitor (Ammenwerth et al, 2007). Each of these approaches has a distinct focus, and its own strengths and weaknesses.

In their seminal review paper, DeLone and McLean defined IS success in terms of success in six categories. These categories were system quality, information quality, user and user satisfaction, individual impact and organizational impact (DeLone and McLean, 2003). In 2003, they revised their model to include service quality as another quality category and they united individual and organizational impact into one “net benefits” category (DeLone and McLean, 2003).

Proper evaluation and assessment is essential for the ongoing improvement of information systems. Yet, a great number of studies that evaluate the organizational aspects of hospital information systems (hereafter HIS) are based on exploratory methods (i.e. do not test a pre-defined hypothesis) (Ammenwerth and Keizer, 2005). Objective and explanatory measurement methods are needed to better evaluate organizational aspects of IS, both in general IS and in HIS context. Another deficiency of the IS evaluation studies is that the evaluated system usually has a limited focus (Ammenwerth and Keizer, 2005), either in functionality or localization (i.e. departmental systems). Among the explanatory studies that utilize formal assessment and/or measurement data rather than user feedback or satisfaction, there are very few studies that have a wider focus and evaluate enterprise-wide systems (Ammenwerth and Keizer, 2005)