This chapter seeks to address the dearth of practical examples of research in the area by proposing that critical realism be adopted as the underlying research philosophy for enterprise systems evaluation. We address some of the implications of adopting such an approach by discussing the evaluation and implementation of a number of automated performance measurement systems (APMS). Such systems are a recent evolution within the context of enterprise information systems. They collect operational data from integrated systems to generate values for key performance indicators, which are delivered directly to senior management. The creation and delivery of these data are fully automated, precluding manual intervention by middle or line management. Whilst these systems appear to be a logical progression in the exploitation of the available rich, real-time data, the statistics for APMS projects are disappointing. An understanding of the reasons is elusive and little researched. We describe how critical realism can provide a useful “underlabourer” for such research, by “clearing the ground a little ... removing some of the rubbish that lies in the way of knowledge” (Locke, 1894, p. 14). The implications of such an underlabouring role are investigated. Whilst the research is still underway, the article indicates how a critical realist foundation is assisting the research process.
Many recent articles from within the information systems (IS) arena present an old-fashioned view of realism. For example, Iivari, Hirschheim, and Klein (1998) see classical realism as seeing “data as describing objective facts, information systems as consisting of technological structures (‘hardware’), human beings as subject to causal laws (determinism), and organizations as relatively stable structures” (p. 172). Wilson (1999) sees the realist perspective as relying on “the availability of a set of formal constraints which have the characteristics of abstractness, generality, invariance across contexts.” (p. 162)
Fitzgerald and Howcroft (1998) present a realist ontology as one of the foundational elements of positivism in discussing the polarity between hard and soft approaches in IS. Realism is placed alongside positivist and objectivist epistemologies and quantitative, confirmatory, deductive, laboratory-focussed, and nomothetic methodologies. Such a traditional view of realism is perhaps justified within the IS arena as it reflects the historical focus of its use; however, there now needs to be a greater recognition of the newer forms of realism—forms of realism that specifically address all of the positivist leanings emphasised by Fitzgerald and Howcroft (1998). A particular example of this newer form of realism is critical realism. This modern realist approach is primarily founded on the writings of the social sciences philosopher Bhaskar (1978, 1979, 1986, 1989, 1991) and is peculiarly European in its origins.
Critical realism is becoming influential in a range of disciplines including geography (Pratt, 1995), economics (Fleetwood 1999; Lawson, 1997), organization theory (Tsang & Kwan, 1999), accounting (Manicas, 1993), human geography (Sayer, 1985), nursing (Ryan & Porter, 1996; Wainwright, 1997), logistics and network theory (Aastrup 2002), and library science (Spasser, 2002). Critical realism has been proposed as a suitable underlabourer for IS research (Dobson, 2001, 2002; Mingers, 2001, 2002), yet there have been few practical examples of its use in IS research. The application of critical realism within the IS field has been limited to date. Mutch (1999, 2000, 2002) has applied critical realist thinking in the examination of organizational use of information. In so doing, he comments how difficult it is to apply such a wide-ranging and sweeping philosophical position to day-to-day research issues. Mingers (2002) examines the implications of a critical realist approach, particularly in its support for pluralist research. Dobson (2001, 2002) argues for a closer integration of philosophical matters within IS research and suggests a critical realist approach has particular potential for IS research. Carlsson (2003) examines IS evaluation from a critical realist perspective. This chapter seeks to address the dearth of practical examples of critical realist use in IS by proposing the review of APMS implementation from such a perspective.