The data created as a byproduct of the functioning of a learning management system (LMS) have been made available to administrators of LMSes through multiple channels on Instructure's Canvas LMS. One of these channels is the packaged “Reports” function in the Admin section, which enables users to download data tables based on formal terms of the academic calendar (all terms, fall, spring, summer, and others). This work explores some highlights from select extracted eras (time periods) of a live LMS instance at Kansas State University. This chapter includes the first term out of the gate for the LMS, public courses and recently deleted ones during the fall/spring/summer sessions during the LMS lifespan, learning tools interoperability (LTI) reports in the LMS instance, competencies, and other insights. Various contemporary data analytics methods are applied to extract meanings from this time-based data.
TopIntroduction
Those who use learning management systems (LMSes) to teach at a distance have a range of data available to them in terms of their learner behaviors, the efficacy of their online assessments, and other common educational data. They can review messaging on discussion boards; they can drop in on student groups and evaluate learner assessments of each other’s works. In recent years, LMS-makers have enabled an even wider level of access to system administrators, including access to big data data portals and reports. Reports enable access to pre-configured data (based on the set study terms in an academic calendar and / or “all terms” for the select data from the entire lifespan of the LMS instance) from the LMS instance for some practical and analytical uses. The formal report names are as follows (in alphabetical order): Course Storage, Grade Export, LTI (Learning Tools Interoperability) Report, Last Enrollment Activity, Last User Access, MGP Grade Export, Outcome Results, Provisioning, Public Courses, Recently Deleted Courses, SIS Export, Student Competency, Students Access report, Students with No Submissions, Unpublished Courses, Unused Courses, User Access Tokens, and Zero Activity.
In the Instructure Canvas LMS instance used at Kansas State University, the available reports begin in Fall Semester 2013 and continue through the present. After in-depth research to see what LMS would best meet the needs of the university and its learners, K-State stood up its version of the new LMS and then transitioned the university over a multi-year period.
From this reports data, it is possible to explore some pre-configured data by micro “eras” of the LMS instance. For example, it is possible to acquire a sense of the system during the first term of the LMS’s activation. It is possible to capture data from all Fall Semesters in order to identify patterns. It is possible to analyze changes over time, such as year-over-year. It is possible to capture all the data and look at a slice-in-time (although this would be easier done with data portal data) (Hai-Jew, Spring-Summer 2017).
The information is downloaded in data tables. While these include personally identifiable information (PII), and certain discrete assertions may be made about individuals, reports are not the most effective for very specific individual-related queries. Back-end data is not particularly helpful to understand the experience of an online course; the front-end of the LMS is better for that. However, as one data stream that may enhance awareness of an LMS, reports can be highly insightful (Hai-Jew, Fall 2017 – Winter 2018). This work provides a sense of some of what may be knowable from the Reports feature in Canvas.
Delimitations
How an LMS is stood up and with what options will affect how it functions and what log data may be available. This work is specific to one live LMS instance. Some of the insights here may be applicable to other contents but not likely all.
Available data may be analyzed in different ways. There has been years of various types of educational data mining (Romero & Ventura, 2007), to inform practices. Central tendencies and statistical moments may be captured for grade sets, for example. Text sets may be analyzed for frequency counts, extant themes, and zoomed-in understandings (such as through word trees). Written expressions may be analyzed for sentiment analysis and auto-extracted themes. Decision trees may be drawn up to understand what factors affect learner performance (what populations learners end up in). Social network graphs may be drawn to show people’s intercommunications and interactions with others. In other words, the data may be analyzed in various ways, and value may be added to the raw data. In this chapter, the focus is not on the various types of applicable analytics, but more on the available Reports data and what may be understood after light data cleaning and some simple queries. The focus is more on the Reports and the time framing based on time periods.
Also, an LMS has a wide range of uses beyond for teaching catalog courses to students. There are non-credit “short” courses built on the LMS substructure. Course shells in an LMS may be set up for a number of reasons: counseling advisement, committee work (hiring, grievance, tenure review, mid-tenure review), program reviews, collaborations (curricular designs, research, grant-writing, report-writing, policy-making, and work), draft courses, training courses (both for internal and external usage), technology testing, clubs (student clubs, book clubs, and others), events planning (homecoming), recitation spaces, intranets, and others.