Mixed Research and Online Learning: Strategies for Improvement

Introduction

Research on online learning has largely been characterized as being low quality (Bernard et al., 2004). Part of the reason for this classification is the near obsession of past researchers with conducting comparison studies; that is, studies that compare online learning to traditional face-to-face learning. Despite researchers’ calls to conduct research with other types of designs, use of comparison studies have been increasing since the mid 1980s (Bernard et al.). In general, researchers have been preoccupied with demonstrating that online learning is as good as face-to-face learning (Wray, Lowenthal, Bates, & Stevens, 2008). However, in the past few years, comparison studies have come under increased scrutiny (Bernard et al.; Meyer 2004). This is because the majority, if not all, of comparison studies—like most research on online learning—have failed to employ robust research designs or control for extraneous variables (Bernard et al.; Meyer; Phipps & Merisotis, 1999).

Even when researchers have had the foresight to avoid conducting comparison studies, they have often overly relied on survey data (Goldman, Crosby, Swan, & Shea, 2005), or other limited types of data collection and analysis. While survey data, which is self-report data and the most often utilized method to study online learning (Hara, Bonk, & Angeli, 2000), is useful and has its place in educational research, this type of data alone is retroactive and insensitive to changes over time (Kramer, Oh, & Fussell, 2006). Therefore, this type of data is not appropriate to investigate all research problems. Even when researchers have chosen not to rely on self-report data and instead to analyze what is said and conducted online, researchers have, for the most part, focused solely on the frequency of participation (Henri, 1992). Commonly, these frequency counts are then analyzed through the use of content analysis (De Wever, Schellens, Valcke, & Keer, 2006). Content analysis is arguably the second most popular type of analysis for studying online learning and the primary method used to analyze online discourse (De Wever, Schellens, Valcke, & Van Keer, 2006). While content analysis is a useful type of analysis, it too cannot, and should not, be used to answer all research questions (Berelson, 1952; Tesch, 1990).

Instead, researchers need to begin to employ different ways of studying online learning that will increase the rigor of the research results. Design based research is one increasingly popular approach that will likely strengthen some of the research conducted on online learning (see Akilli, 2008; Joseph, 2004; Kelly, 2004; Reeves, 2005; Reeves, Herrington, & Oliver, 2004, 2005). However, there are some important differences between design based research and mixed research (Akilli, 2008). Thus, in addition to conducting design based research, researchers should begin to employ mixed approaches to study online learning. While some researchers of online learning have argued for the importance of using multiple methods when studying online learning (Goldman, Crosby, Swan, & Shea, 2005; Gunawardena, Lowe, & Anderson, 1997; Hiltz & Arbaugh, 2003), the majority of research conducted on online learning currently is mono-method.