Human Systems Engineering and Educational Technology

Human Systems Engineering and Educational Technology

Rod D. Roscoe (Arizona State University, USA), Russell J. Branaghan (Arizona State University, USA), Nancy J. Cooke (Arizona State University, USA) and Scotty D. Craig (Arizona State University, USA)
Copyright: © 2018 |Pages: 34
DOI: 10.4018/978-1-5225-2639-1.ch001
OnDemand PDF Download:


The design and development of educational technologies is a complex, interdisciplinary endeavor. Learning science research reveals principles of learning and instruction, and advances in computer science implement these principles in innovative technologies. This chapter promotes a complementary discipline—human systems engineering or “user science”—that emphasizes designing with human users' goals, needs, capabilities, and limitations in mind. Systematic and iterative human systems engineering should contribute to educational technologies that are more functional, usable, desirable, and ultimately more effective. The authors overview key human systems engineering principles (e.g., usability and user experience) and methods (e.g., cognitive task analysis, contextual inquiry, heuristic evaluation, and participatory design), and then consider example applications from research on automated writing evaluation technologies. The chapter concludes with broad research questions posed to researchers, developers, and educators in the field of educational technology.
Chapter Preview


The design and development of educational technologies is a complex, interdisciplinary endeavor. For instance, learning science research can reveal principles of learning and instruction, such as comprehension processes (Chi, 2000; McNamara, 2004) and human tutoring (Chi, Siler, Jeong, Yamauchi, & Hausmann, 2001; Graesser, Person, & Magliano, 1995; VanLehn, Siler, Murray, Yamauchi, & Bagget, 2003). Advances in computer science then enable the implementation of these principles in innovative technologies, such as intelligent tutoring systems that teach self-explanation (McNamara, Levinstein, & Boonthum, 2004) or physics (Graesser et al., 2004; VanLehn et al., 2005). The most successful and impressive educational technologies tend to emerge from the integration of multiple approaches.

One question is whether such work addresses the full scope of human users’ (e.g., students and teachers) needs. A typical approach for educational technology research is to first develop a functional system and then evaluate it in lab or classroom studies. These tests include measures of learning or other growth, and may incorporate student perception or reaction data. This information is useful for pinpointing flaws to repair, and can sometimes explain mixed results (e.g., discovering that scaffolding hints were not grade-level appropriate). However, identifying problems may be possible earlier and less expensively via iterative usability testing. For instance, a handful of students might be asked to read and explain potential hints before the scripts are ever coded into the system. If students stumble or express confusion, revisions could make the text more readable. A “failed” study could be avoided.

In this review-style chapter, we promote a third research discipline that complements learning science and computer science, but which appears underrepresented in educational technology. This discipline—human systems engineering (or “user science”)—entails research and design that takes into account human users’ broad goals, needs, capabilities, and limitations. Notably, some “human factors” have been addressed across decades of learning science and computer science. For example, researchers have examined learners’ achievement goals (Harackiewicz, Barron, Pintrich, Elliot, & Thrash, 2002), feedback needs (Shute, 2008), prior knowledge (Shapiro, 2004), and misconceptions (Chi, Roscoe, Slotta, & Chase, 2012). Similarly, artificial intelligence and learning analytics advances (Baker & Yacef, 2009; Berland, Baker, & Blikstein, 2014; Desmarais & Baker, 2012) enable technologies that adapt to learners’ knowledge and performance (Aleven, McClaren, Sewall, & Koedinger, 2009; VanLehn, 2006), strategies (Winne & Hadwin, 2013), and emotions (Calvo & D’Mello, 2010; Woolf et al., 2009).

To the extent that users of these technologies are “learners” and “teachers,” any work that supports learning and teaching can be considered “user centered.” However, there are aspects of user needs that go beyond instruction. A central assumption is that systematic and iterative human systems engineering can contribute to educational technologies that are more functional, usable, and desirable, ultimately resulting in systems that are more effective. We first introduce human systems engineering along with key principles (e.g., usability; Nielsen & Budiu, 2013) and methods (e.g., knowledge elicitation; Cooke, 1994). To make these concepts more concrete, we then discuss examples from the development of automated writing evaluation systems.

Complete Chapter List

Search this Book: