Growing enrollments in online learning have dramatically increased the number of remote, adjunct faculty teaching online. This is a challenge for universities to create scalable strategies to develop, evaluate, and support faculty in the online classroom. While robust qualitative and quantitative faculty performance metrics exist, faculty analytic data is typically difficult to retrieve and even more difficult to efficiently analyze. This chapter overviews the value of key performance indicator dashboards (KPI) that can help automate the collection and use of faculty analytic data to enhance faculty development and, ultimately, foster positive student learning experiences.
TopIntroduction
The rapid growth of online education (Allen & Seaman, 2014) requires higher education institutions to find scalable ways to monitor, evaluate, and support teaching to ensure instructional quality in the online classroom (Mandernach & Palese, 2016). While faculty teaching online have clearly expressed a need for instructional support, they often do not seek help and try to remedy issues with self-made solutions that are ineffective or do not follow institutional policies (Ferencz, 2017). As such, it is essential that universities provide dedicated faculty support to ensure high quality online instruction that fosters student learning (Ferencz, 2017). Unfortunately, traditional reliance on postinstruction indicators (such as student evaluations, student performance metrics, or reflective classroom evaluations) fails to provide the necessary support to identify faculty who need assistance while they are actively teaching. The problem is compounded when considering that traditional evaluation metrics are often unable to scale to meet the demands of growing online enrollments. The use of key performance indicator (KPI) dashboards provides an opportunity for: 1) timely identification of instructors who need additional support; 2) targeting professional development resources to meet individual faculty needs; and 3) highlighting effective faculty and instructional practices.
Teaching online produces a wide range of data that can provide insights into teaching strengths and weaknesses. The data collected by learning management systems (LMSs) allows for the collection of quantitative metrics across a wide range of online teaching behaviors. These metrics can be used to generate reports and create automated workflows to highlight and correct desired teaching behaviors. For example, best practices in online education highlight the importance of instructor interaction and feedback in the online classroom. These best practices may be reflected in teaching behaviors such as frequency of discussion posts, number of announcements, or timeliness of assignment grading and feedback. While these indicators do not provide a complete picture of instructor performance, they provide insight into baseline instructor performance. Other metrics, such as student success rates and end of course surveys (EOCS), provide insight to understand the overall impact of faculty behaviors. As highlighted by Lyde, Grieshaber, and Byrns (2016), multiple measurements can provide a more accurate depiction of online classroom performance. By combining the immediate metrics available via the LMS with postinstruction indicators of performance, institutions can more efficiently and effectively target professional development resources to the faculty that need it.
Typically, institutions identify classroom instruction concerns only after a course ends—either via poor end-of-semester evaluations or student complaints. By default, traditional reliance on postcourse evaluation strategies means that instructors who are struggling may not receive support in time to benefit the students they are currently teaching. This issue can be mediated via the use of analytic data. Integration of analytic measures utilizes real-time data to streamline the process of identifying and correcting instructional quality issues before they affect student outcomes (Wagner, 2014). As highlighted by Marks, AL-Ali, Majdalawieh, and Bani-Hani (2017), technology can improve timely quality measures through the analysis of quantitative classroom data. Using real-time systems data (i.e., the LMS, EOCS, faculty management databases, etc.), faculty professional development teams can: 1) identify instructors who fail to meet baseline performance standards; 2) provide targeted professional development based on an instructor’s individual performance, and 3) determine if an instructor needs be qualitatively peer-reviewed. While most universities have access to this type of real-time data, it is often housed in many different sources (such as the LMS, evaluation databases, and faculty and student database management systems) and assembling this data takes time and resources. Simply put, the challenge is not in the collection of data but rather in the development of a functional system that allows for timely, efficient use of data to make actionable changes to support faculty.