Educational technology development is a design problem. Product developers must optimize between what educational research suggests would be most effective, technological or other software development constraints, and the practical needs of end users and key stakeholders. Creating a logic model and using it to guide a user research program can help product developers tackle this problem. A logic model is a structured description of how a specific product achieves an intended learning outcome. Developing a logic model helps product developers make explicit their assumptions about users, product features, and use cases. Then a user research program can be constructed to test each of these assumptions and provide actionable feedback for further iterations of the product. In this chapter, we present three cases that highlight how the logic model approach can guide a program of research, and how that research has led to tangible product improvements.
TopIntroduction
Educational technology development is a complex design problem. Product developers must optimize between what educational research suggests would be most effective, technological constraints, and the practical needs of the end user. In addition, educational technology operates within a complex ecosystem that contains multiple types of users and stakeholders (i.e., students, teachers, administrators, and parents), different physical learning environments, and different expectations for different use cases (e.g., a curriculum product that aims to improve scores on high-stakes standardized tests vs. an educational game that children play outside of the classroom).
WestEd has partnered with over 50 educational technology companies to conduct iterative user research for products at different stages of development. These products include educational games, student-facing instruction, and teacher tools. User research at WestEd is a collaborative, interdisciplinary process that brings experts in academic content, teaching, learning sciences, and human-computer interaction together with the product developers. Developing an effective educational technology product requires an understanding of all these fields.
To tackle the educational technology design problem, we adopt a specific iterative user research approach that is centered around a product’s logic model. A logic model is a structured description of how a specific product achieves an intended learning outcome. The focus of the logic model is on precisely describing the mechanism behind the product’s effects. The use of logic models is common in academic research and program evaluation, but is less common in technology product development. Table 1 shows an example of a logic model for a fictional educational game in mathematics.
Table 1. An example of a logic model for a fictional educational game in mathematics
| Inputs | Activities | Outputs | Outcomes | Impacts |
• Socioeconomically disadvantaged students ages 10-12
• Game provides opportunities to explore math operations in non-threatening environment
• Game provides immediate feedback on student moves | • Students play game independently at home for at least 10 minutes, 3 times a week, for four months
• Students encode feedback from game | • Students reorganize mathematics knowledge
• Students experience enjoyment during game-play | • Students adopt positive attitudes and increased confidence in mathematics
• Students exhibit improved scores on tests of mathematical problem solving | Students exhibit improved grades and retention in mathematics |
WestEd’s approach to logic models is a variant on the five-column logic format that describes the inputs, activities, outputs, outcomes, and impacts of a program (W.K. Kellogg Foundation, 2004).