Piloting Artificial Intelligence (AI) to Facilitate Online Discussion in Large Online Classes: A Case Study

Introduction

The purpose of this case study was to explore the faculty experience and perception of AI-driven platforms for teaching online classes. The inspiration for this research was twofold. The researchers, as instructional designers and online learning experts, received requests from faculty about using AI discussion tools to manage large class online discussions. There was also an emerging research need to gain knowledge about using AI-driven tools for instructional design in hybrid and online learning. In courses facilitated with online environments, asynchronous discussion is a common and valuable learning activity. In these courses, especially with large enrollment, instructors and students felt challenges to effectively manage, facilitate, and engage with asynchronous online discussion due to the high volume and quality of responses (Hew & Cheung, 2014).

With the shared understanding that these are needed to ensure the achievement of learning objectives, the researchers collaborated with faculty at a higher education institution in the USA to identify ways to maximize the instructional benefits of online discussion. One of these efforts was to investigate possible tools or technologies to make the facilitation demands more manageable for faculty as well as appraise the implications of newer tools to teaching and learning. Following the call to better understand and provide empirical evidence of AI tools, the researchers approached this case study with exploratory methods and collaborative reflections of four faculty who used Packback for their classes.

The COVID-19 pandemic required the researchers’ institution to pivot teaching to mostly online format. One of the major concerns shared by faculty at this institution centered on ensuring interactivity and engagement online. A third-party online discussion tool called Packback that uses algorithms derived from artificial intelligence (AI) became a viable option to “address issues of engagement and meaningful evaluation, which [was] found to be lacking when using traditional learning management system (LMS) forum tools” (Butcher et al., 2020, p. 381). Packback uses artificial intelligence detecting superficial inquiries from students, and machine learning mechanism, based on learners’ intrinsic motivation and self-determination theory to drive learners’ curiosity (Butcher et al., 2020; packback.co). Packback approaches online discussion using a “curiosity score” that is based on length of posts, sentence structure, and presence of a source citation, amongst other variables. Grades can be calculated with curiosity scores (Heap et al., 2020). However, Heap et al. (2020) also argue that despite the growth in popularity of AI platforms, there is a lack of empirical studies particular in higher education.

Packback (https://www.packback.co/) uses AI to provide real-time automated feedback and moderation of student discussion responses. It also uses a machine learning algorithm to encourage a “social networking site” kind of “popularity” among peers, by the differentiation between instructors’ private coaching along with public praise. The AI tool is designed to encourage students to ask thoughtful questions. This automation gives the instructor more time to provide more focused and higher-level feedback to student responses. From a design perspective, this mechanism is supposed to more deeply facilitate learning in the course discussions. With Packback, students can share their responses to discussion prompts and the AI gives individualized helpful feedback on superficial responses to encourage deeper thinking and responses. With these features, Packback is assumed to be useful for increasing discussion activity and feedback in large enrollment courses. With this tool, a faculty member appears to be able to receive computer-assisted help with facilitating the conversations. However, there is still ambiguity, confusion, and concerns in terms of student perceived usefulness, the roles and functions of faculty intervention in this AI-driven platform, and how instructional design and systematic integration of the tool can mobilize the effectiveness of teaching and learning (Heap et al., 2020). There are needs to understand how these AI-facilitated online discussion platforms work and how teaching strategies and student learning experience can be affected.