Health Information Technology (IT) implementation can fail or meet high levels of user resistance for a variety of reasons, including lack of attention to users’ needs and the significant workflow changes induced and required by the technology. End-user satisfaction is a critical factor in health IT implementation. In this paper, the authors describe the process of developing and testing a questionnaire to evaluate health IT implementation, in particular Computerized Provider Order Entry (CPOE) and Electronic Health Record (EHR) technologies. Results show evidence for the validity and reliability of the questionnaire. The Systems Engineering Initiative for Patient Safety (SEIPS) questionnaire is easy to administer and allows researchers to evaluate different aspects of health IT implementation. Results of this research can be used for benchmarking results of future studies evaluating health IT implementation.
TopIntroduction
One of the important applications of information technology (IT) to health care is Electronic Health Record (EHR), which includes the functionalities of computerized provider order entry (CPOE) and electronic medication administration (eMAR) (Lee, Teich, Spurr, & Bates, 1996; Sittig & Stead, 1994; Tierney, Miller, Overhage, & McDonald, 1993). In CPOE, physicians and other providers enter orders directly into the computer instead of using a paper-based system. Through rapid information retrieval and efficient data management, CPOE systems have the potential to improve quality of patient care (Bates, et al., 1998; Bates, et al., 1999; Murff & Kannry, 2001; Overhage, Tierney, Zhou, & McDonald, 1997; Teich, et al., 2000). For reviews of the effects of CPOE on medication safety and quality of care, see Kaushal et al. (2003) and Kuperman et al. (2007). There are four specific areas in which CPOE can deliver specific advantages over traditional paper-based systems: process improvement, resource utilization, clinical decision support and guideline implementation (Kuperman, et al., 2007; Murff & Kannry, 2001; Sittig & Stead, 1994). According to some experts (Bates, Kuperman, & Teich, 1994; Lee, et al., 1996), a major advantage of CPOE and other EHR functionalities such as eMAR is the opportunity to receive online support information at the point of care. Despite the benefits of EHR (Kaushal et al., 2003), many attempts to implement such systems have failed or met with high levels of user resistance (Bates, 2006; Bates, et al., 1994; Connolly, 2005; Lee et al., 1996; Massaro, 1993a, 1993b; Sittig & Stead, 1994).
EHR implementation efforts have stumbled for a variety of reasons, including lack of sensitivity to users’ needs and the significant changes induced and required by the technology (Massaro, 1993a, 1993b). Only a relatively small percentage of hospitals use EHR. Results of a survey in 2002 (Ash, Gorman, Seshadri, & Hersh, 2004) showed that CPOE was not available to physicians in 84% of the hospitals; completely available in 10% of the hospitals and partially available in 6%. More recent estimates suggest that EHR usage is increasing, but most hospitals are still in the planning stage (Delbanco, 2006; Jha, et al., 2009).
As part of a large study funded by AHRQ (http://cqpi.engr.wisc.edu/cpoe_home), we are evaluating EHR implementation in several intensive care units (ICUs) of a large hospital using a variety of methods, including observational methods, interviews with key personnel, focus groups, a survey questionnaire, assessment of medication errors and adverse drug events, and various quality of care indicators. In this paper we focus on the survey questionnaire. The EHR being evaluated include various functionalities: CPOE, eMAR, physician and nursing documentation, and nursing flowsheets.