Do Student-Written Responses to Reflection Questions Predict Persistence and Performance in Online Courses?: A Text Analysis Approach

Introduction

Over the last decade, online course offerings have become a standard at most colleges and universities (Cohen & Soffer, 2015; Lee, 2016). The growth in online enrollments is more than ten times higher than the growth in overall higher education enrollments, and this trend is expected to continue (Seaman, Allen & Seaman, 2018). Concomitant with the boom in online learning, there are escalating concerns about academic accountability, specifically student outcomes as measured by persistence and success (Boston & Ice, 2011; Hachey, Conway & Wladis, 2013; Jaggars & Xu, 2016). This arises from research that indicates that attrition rates in online courses are significantly higher in comparison with face-to-face courses (Hachey et al., 2013; Jaggars & Xu, 2016; Patterson & McFadden, 2009). A growing body of research suggests that the high online dropout rate is the result of lack of certain personal attributes and self-regulation skills, which are essential for success in online courses (Glick, Cohen, Festinger, Xu, & Warschauer, 2019; Jaggars & Xu, 2016). Self-regulation plays a particularly important role in virtual learning environments, where students are required to take greater responsibility to control and regulate their own learning (Appana, 2008; Duffy & Kirkley, 2003). It is important, therefore, to be able to identify early in the course students who lack self-regulation skills in order to improve retention and graduation.

A growing body of research is seeking to respond to this challenge by employing learning analytics techniques to analyze student online learning behavior, log files, and clickstream data. Such analysis may include flow analysis to view students’ learning journey, analysis of number of video views, or number of minutes’ views (Cohen, Shimony, Nachmias, & Soffer, 2019). However, research examining whether other types of analytics – such as text analysis techniques – can help predict persistence and performance in online courses, is scarce. Therefore, the purpose of this study is to employ text analysis techniques to analyze student reflective responses to a series of skill-building videos incorporated into online courses. Specifically, this study aims to examine whether persistence and performance of online Latin American students may be predicted by evidence of significant learning found in student written responses to post-video reflection questions.

This study contributes to the literature on persistence in online learning in several important ways. First, this study performs what appears to be the first attempt to use text analysis techniques to examine whether persistence is affected by evidence of significant learning found in student written responses. By answering this question, university leaders and online course administrators will be able to identify at-risk students using a more holistic approach, which integrates learning analytics with text analysis techniques, thus improving graduation by targeting individual students in meaningful ways. Second, course administrators often use midterm scores to predict success in online courses (Glick et al., 2019). Although midterm scores prove to be a strong predictor of students’ final grades, they are collected at a point when students have already finished a significant amount of the coursework. Therefore, more research is needed to explore factors such as text analysis techniques, which may prove effective in identifying at-risk students by analyzing reflective responses submitted early in the course. Finally, while a growing number of institutions across Latin America are transitioning from brick-and-mortar to online teaching (Glick et al., 2019; Xu et al., 2019), many Latin American students struggle in such settings. Therefore, to improve persistence and performance in online courses implemented in developing countries, it is important to examine whether Latin American students who are at-risk of dropping out can be identified using text analysis techniques – a previously understudied but critically important approach.