This chapter reports on ongoing design-based research being conducted in the fully online Master of Arts in Teacher Leadership (MTL) program at the University of Illinois Springfield. After an initial Quality Matters (QM) review and redesign, semester-to-semester implementation issues identified by the Community of Inquiry (CoI) survey form the basis for ongoing course improvements. Preliminary results from a four-semester iterative redesign of one course demonstrated the efficacy of using this design-based process for significantly improving learning outcomes. Currently, departmental faculty are collaborating on the redesign of all core courses in the MTL program using a similar process guided by an initial QM redesign followed by iterative and incremental changes driven by CoI survey results.
TopBackground
This chapter reflects on our experiences in using an iterative, collaborative, theory-based and data-driven approach to improving online courses based on student perceptions of social, teaching and cognitive presence. Across several semesters of recursive modifications to online graduate courses in the MTL program, we came to situate our improvement efforts within a design experiment perspective. Specifically, we applied a design-based approach to fully online courses and used the Community of Inquiry (CoI) framework as a theoretical compass to guide iterative course modifications.
We are not the first to use the Community of Inquiry framework as a conceptual guide for program-wide change (see, for example, Bogle, Cook, Day, & Swan, 2009, and Vaughan & Garrison, 2006). Our project builds on that earlier program-level work, with some important implementation and methodological differences. The redesign project described in this chapter is within a fully online program; the work described by Bogle, Cook, Day and Swan and by Vaughan and Garrison was done within blended programs. In addition, the redesign project we describe in this chapter was a faculty-initiated grassroots effort by program faculty, with no university-wide initiative providing a catalyst for this ongoing course improvement work. Perhaps most importantly, our work is guided not just conceptually by the Community of Inquiry framework; it is directly and repeatedly informed by course-level CoI survey data. We thus use theoretically-derived empirical evidence from our online courses to guide their redesign.
When we began the project, we had planned no further than to do what we assumed would be a one-time, quasi-experimental pre-post study in the fall semester of 2009 to explore the effects of a single course redesign. The initial revisions made were based on a Quality Matters (Quality Matters, 2005) review of an Educational Research Methods course. We sought to measure the effects of the revisions, not just on learning outcomes, but also on learning processes as measured by the Community of Inquiry (CoI) survey (Swan, Richardson, Ice, Garrison, Cleveland-Innes, & Arbaugh, 2008). We imagined that an improved course design would result immediately in improved learning processes and that improved learning processes would, in turn, support improved student outcomes.
Quality Matters Rubric
Quality Matters is a faculty-oriented, peer review process designed to assure quality in online courses and structured around a rubric (Appendix A) based on instructional design principles (Quality Matters, 2005; Shattuck, 2007). The rubric consists of 41 standards which are assigned point values of 1, 2, or 3, and are organized into eight categories -- course overview, learner objectives, assessment and measurement, instructional materials, learner interaction and engagement, course technology, learner support, and accessibility. Three reviewers assess whether or not a course meets or does not meet each of the standards. All three point standards, of which there are 21, as well as a minimum score must be met. Nine of the 21 essential (3 point) standards involve learning objectives.