Academic Assessment of Critical Thinking in Distance Education Information Technology Programs

Introduction

While the growth of distance education programs has been significant in the past two decades, the assessment of student learning in online schools is emerging. The topic of assessment is important because in the last few years, academic assessment has become an integral part of student learning at traditional and online institutions (WASC, 2013; Miller, 2012). From the perspective of accrediting bodies, an assessment process must be implemented to demonstrate continuous improvement in the quality of teaching programs. In the context of globalization, American Universities teaching abroad also transfer the assessment process and best practices to their foreign branches (Noori & Anderson, 2013). The role of faculty is critical through taking ownership, developing strategies, and ensuring that educational goals are met and constantly refined. When assessment methods and techniques are incorporated into the classroom, student learning is evaluated and measured against a set of learning outcomes. Formative and summative assessments are used to collect data and compare results against some form of metrics or benchmarks. Formative assessments are used to capture students’ knowledge at some point in time during class whereas summative are conducted at the end of the course or program of study. These evaluations also detect possible gaps and opportunities to drive course improvement and learning activities. Assessment is a continuous process improvement that should be aligned with the institution’s mission to provide an exceptional educational experience for students (Walvoord, & Banta, 2010, p.27). This outcome-based approach equally benefits faculty and students by institutionalizing a culture of instruction and learning.

An important element in assessment is Learning Outcomes (LOs). These are defined as educational objectives concisely articulated to convey the fundamental learning and skills that students should achieve during a course or program of study (WASC, 2013, p.12). LOs must be task specific and measurable. They are widely recognized as the prerequisite for a “learner centered” approach. To measure LOs, rubrics are among the most common direct methods used to collect assessment data and evaluate student learning. There are a variety of rubric designs, all of which are meant to inform mastery of skills or areas needing improvement. The Association of American Colleges and Universities (AAC&U) has been recognized as the leader developer of rubrics. The association has designed a series of Valid Assessment of Learning in Undergraduate Education (VALUE) rubrics for the purpose of measuring student achievement in 15 intellectual and practical skill areas of study (AAC&U, 2014).

To emphasize the application of rubrics to learning outcomes, this chapter discusses a university assessment process in terms of signature assignments selected for the Computer Science (CS) and Information Technology Management (ITM) programs. The university is an online institution granting undergraduate, masters, and Ph.D. degrees. During the assessment planning, a curriculum map was prepared to determine alignment between courses and specific Institutional Learning Outcomes (ILOs). Within the ILOs to be measured, the critical thinking ILO was chosen since it was the academic theme for the year. Signature assignments were then carefully selected by course to ensure that critical thinking could be measured at the introduced, reinforced, or emphasized levels. The assessment results are discussed as evidence that critical thinking abilities can be assessed through rubrics and across the first and last course of each program. The findings also indicate that relevant gains were obtained from students attending the emphasized level courses when compared to the students sampled in the introduced level course. The study methodology is explained in this chapter.

This chapter develops some standard practices in the assessment process and presents conclusions about the advantages and challenges of the assessment tools and practices from the perspective of faculty and University administration.

The authors of the chapter have the experience of serving in the Assessment Committee of the University and serving as Program Chair of these academic programs.