Abstract
The measurement of online latent constructs, such as student engagement, have mimicked the measurement of these constructs in traditional, brick-and-mortar learning environments. For brick-and-mortar K-12 schools and online K-12 schools there are challenges that need to be addressed to establish a measure with support for validity. Measurement in an online learning environment has different accessibility and data collection requirements. The online learning environment lends itself well to the use of student behaviors to measure latent constructs, including student engagement. The behaviors of students in an online learning environment are regularly documented. This chapter will delve into the ways latent constructs, using student engagement as an example/case study, are currently being measured in the K-12 online learning environment and alternatives to these measures mimicking traditional brick-and-mortar measures.
TopIntroduction
The measurement of online latent constructs, such as student engagement, have mimicked the measurement of these same latent constructs in traditional, brick-and-mortar learning environments. For brick-and-mortar K–12 schools and online K–12 schools there are challenges that need to be addressed to establish a measure with support for validity. One of the most elusive latent constructs in K–12 education, including online K–12, is student engagement. Several studies have assessed student engagement, both in an online learning environment and in other learning environments (Chen, Lambert, & Guidry, 2010; Hamane, 2014; Lerma, 2010; McNeal, Spry, Mitra, & Tipton, 2014). Yet there are still challenges in the measurement of student engagement. Fredricks and McColskey (2012) stated “one of the challenges with research on student engagement is the large variation in the measurement of this construct, which has made it challenging to compare findings across studies” (p. 763). If latent constructs, such as student engagement, are not consistently measured, then the impact of these constructs on academic achievement is not clear.
In the last 30 years there has been an influx of K–12 students into the online learning environment (Allen & Seaman, 2010; Hamane, 2014; Parker, 2015). These online students range from students using technology-mediated learning in traditional brick-and-mortar schools, attending blended programs where content and courses are online to 100% online students who attend an online/virtual school. More research needs to be done to fully understand the differences between traditional K–12 brick-and-mortar learning environment students and K–12 online learning environment students. These differences can require differences in assessments of academic achievement as well as differences in the measurement of latent constructs. As explained by Pazzaglia, Clements, Lavigne, and Stafford (2016), “although research and recommendations exist for promoting student engagement in face-to-face school settings, it is unclear whether those recommendations translate to the online learning environment, because online settings involve unique considerations and challenges” (p. 3).
Latent constructs are the bread and butter of the educational research community (Samuelsen, 2015). A latent construct is a factor that cannot be directly measured or observed but must be indirectly measured using one or more observable indicators of the latent construct. Student engagement is an example of a latent construct. While student engagement has been found to be important to academic achievement (Betts, 2012; Cremascoli, 2011), college preparedness (Allen & Seaman, 2010) and other academic factors (Appleton, 2017), student engagement is not able to be directly observed or measured without the use of observable indicators of student engagement.
In order to find and/or select observable indicators that can be used to measure latent constructs, such as student engagement, the latent construct must first be able to be defined and operationalized. Currently, there is not a clear definition for student engagement (Appleton, 2017; Fredricks & McColskey, 2012) nor for many other vital latent constructs. This makes the measurement of these latent constructs in a K–12 online learning environment even more difficult. It is essential when measuring latent constructs, such as student engagement, to take into account individual and contextual differences (Sinatra, Heddy, & Lombardi, 2015). This includes the differences in learning environment and how those differences change the manifestations and measurement of latent constructs (Dixson, 2015; Roblyer & Wiencke, 2004).
Key Terms in this Chapter
Indicator: Observable items/variables that are used in the measure of latent constructs.
Traditional Brick-and-Mortar Learning Environment: A learning environment that is based in a face-to-face setting.
Variance/Variation/Variability: The spread (breadth and depth) of the possible scores/values in a variable’s distribution.
Student Engagement: The cognitive, behavioral, or emotional effort that a student puts into their learning and learning process, whether for a single activity/assignment or for their overall learning experience.
Item Response Theory (IRT): Set of models that uses indicators to measure latent constructs through the assignment of both person ability and item difficulty.
Facilitator: Items/variables that interact and or have statistical relationships with latent constructs.
Latent Construct: A factor that cannot be directly measured or observed; a factor that requires the use of indicators to be measured.
Online Learning Environment: A learning environment that is found in an online or virtual setting, instead of in a face-to-face setting.