Evaluation of Simulation Games for Teaching Production (Engineering)

Introduction

Today’s manufacturing is marked by trends towards globalization and rapid technological advances. This results in manufactured products being increasingly customized and complex with shorter life-cycle times, which increases the marginal cost per product (Scheer, 2002). Therefore, organizations are confronted with the challenge of continuously adjusting their capacities and machines, necessitating a high degree of flexibility in dynamic environments.

In addition to business complexity, a number of behavioral factors come into play and make the challenges, which organizations face, even greater. First, the bounded rationality of the economic actors (Simon, 1997) is a supplemental element, which exacerbates the situation. Indeed, decision makers generally opt for the first satisfying solution, and hence, they stop looking for better alternatives. Second, decision makers, like people in general, are prone to the misperception of feedback. This means that their performance in complex and dynamic systems is hindered by non-linearities, time delays and feedback structures (Sterman, 1989). Therefore, decision makers will tend to make poor decisions. Third, decision making in dynamic systems is hard because it calls for dynamic decision making, that is, where a stream of decisions are interdependent on one another. Last, decision makers are also limited by the magical number seven, plus or minus two (Miller, 1956). This number sets the maximum number of cues, which can be simultaneously considered by people while they evaluate a problem. Consequently, organizations are more and more eager to collaborate around structured and emergent manufacturing frameworks such as production networks. These networks entail the joint-manufacturing of products and are regarded as a new form of co-operation between organizations (Wiendhal & Lutz, 2002). Although, organizations do take advantage of being a part of production networks, since today competition takes place between entire supply chains, or networks, instead of single organizations, production networks are nevertheless vulnerable and inflexible since many disparate entities populate them, increasing the risk of collapse due to external shocks of market instability, or boom & bust cycles.

Dynamic systems such as production networks confront their workforces with ever-changing working environments (Baalsrud Hauge et al., 2006). This stresses the need for continuous learning, which constitutes the true competitive advantage for organizations (Senge, 1990, p. 17). Moreover, the learning rate of the organization must be higher than that of competition, so that the former can survive (de Geus, 1988). An effective tool for mediating learning is serious computer games, also known as business games (Warren & Langley, 1999). Computer games not only convey hard skills such as the understanding of how complex systems operate, such as production networks, but also mediate soft skills, like collaboration and communication (Scholz-Reiter et al., 2002). Even though it is shown that the use of games are useful for mediation of soft skills (Windhoff, 2001), it is still difficult to find suitable methods for measuring the learning outcome of serious games.

The objective of this chapter is to show different approaches for the evaluation of the learning outcomes of serious games and to discuss the advantages and disadvantages of the methods. The chapter is based on the authors’ experience of using games in lecturing activities in the field of product development in production networks. The target group for the games is engineering students. The first section of the chapter deals with the background of using serious games for the mediation of skills, and gives an introduction to the two games. These games have been used in several courses over several years, so there are a lot of evaluation results available. The second section deals with the different evaluation methodologies that were applied. It also explains the different results. The final section is a comparison on the used methods and also includes a discussion of the advantages and disadvantages of the different methods.