Abstract
In this chapter a digital assessment and an associated novel mathematical statistical model are provided as online psychometrics designed to evaluate College and University courses. The psychometric evaluation tool is a Student Ratings of Instruction [SRI] instrument used at a Historically Black College and University [HBCU] for course evaluation purposes. The research methodology is an a posteriori post hoc investigation that examines the reliability and validity of the items used in the SRI instrument. The sample under analysis consisted of the responses to 56,451 total items extracted from 7,919 distributed Student Ratings Instruments delivered online during the 2012 academic year. The post hoc application of the novel Tri–Squared Test analysis methodology is used to intricately analyze the results of an earlier study on SRIs that yielded strong construct validity from Cronbach's Alpha Reliability Model, Goodman & Kruskal's Lambda, and Principal Component Factor Analysis with Varimax Rotation.
TopIntroduction
Throughout the academy there has always been concern about improving the skills and knowledge of students. Many colleges and universities have promoted a multitude of strategies to enhance student success, and are constantly exploring ways that would further improve teaching excellence and student success. In attempting to acquire data regarding teaching efficacy, many universities have, over the years, utilized independent student survey designs to evaluate teaching and determine if student learning outcomes are met. Internationally, educational researcher Richardson noted several significant student evaluations of teaching [“Student Ratings” referred to as “Student Evaluations of Instruction” or “SEI”] in use in research projects in the US, England, and Australia, including the use of the British Noel–Levitz “Student Satisfaction Inventory”; the “Course Perceptions Questionnaire”; the “Student Evaluations of Educational Quality”; and the “Course Experience Questionnaire” (as cited in Skowronek, Friesen, & Masonjones, 2011).
Certainly, of the various assessment designs available to evaluate teaching effectiveness and student success, the student ratings are the most widely used in many universities worldwide because they offer an organized, methodical, and effective means of obtaining feedback on students’ responses to instructors and courses (Agbetsiafa, 2010), and have been around since the mid–1920s (Cohen as cited in Donnonet al., 2010; d’Apollonia & Abrami as cited in Safavi, Bakar, Tarmizi, & Alwi, 2012; Wright, as cited in Gravestock & Gregor–Greenleaf, 2008). In general, there have been some agreements that, students’ ratings seem sufficient to evaluate what they seek to determine: teaching effectiveness, student satisfaction, educational experience, and program curriculum (Abrami as cited in Gravestock et al., 2008; Agbetsiafa, 2010; Beran, Violoto, & Kline, 2007; Skowronek, et al., 2011; Zhao et al., 2012). Gravestock et al. (2008) also noted “the quantifiability and comparability of most course evaluations makes the imprecise art of evaluating teaching seem more objective and manageable” (pp. 10). In addition, Titus (2008) noted that, apart from securing teaching efforts to preferred outcomes, probing students about their knowledge underpins the commitment of classroom efforts and events. Additionally, other academics also agreed that student ratings can be an integral part of the evaluation of an instructor’s performance (El Hassan, 2009; McKeachie & Hofer as cited in Skowronek et al., 2011).
Key Terms in this Chapter
SRI: An acronym for “Student Ratings of Instruction” or “Student Ratings” referred to as “Student Evaluations of Instruction” or “SEI” in use in research projects in the US, England, and Australia, including the use of the British Noel–Levitz “Student Satisfaction Inventory”; the “Course Perceptions Questionnaire”; the “Student Evaluations of Educational Quality”; and, the “Course Experience Questionnaire” (as cited in Skowronek, Friesen, & Masonjones, 2011 ).
Teaching Effectiveness: Involves the effectiveness teaching and the transfer of knowledge. Determining teacher effectiveness may not be uncommon to academics since various studies on teaching excellence and student learning outcomes have already been completed ( Hunsaker, Nielsen, & Bartlett, 2010 ; Perry as cited in Keeley, Furr, & Buskist, 2010 ; Schrodt, Witt, Myers, Turman., Barton, & Jernberg, 2008 ).
Goodman & Kruskal’s Lambda (?): An advanced statistical cross tabulation analysis measure of proportional reduction in error ( Goodman & Kruskal, 1954 ).
HBCU: An acronym for “Historically Black College & Universities” (U.S. Department of Education, 2008).
Cronbach’s alpha: In advanced statistical metrics, Cronbach's Alpha ( Cronbach, 1951 ) is a mathematical coefficient designed to measure internal consistency. In psychometrics it is the most practically used statistic that provides (upon precise calculation) an estimate of the overall reliability of a test designed to measure a specified researchable criterion on a sample of examinees.
Tri–Squared Test: Tri–Squared comprehensively stands for “The Total Transformative Trichotomous–Squared Test” (or “Trichotomy–Squared”). The Total Transformative Trichotomous–Squared Test provides a methodology for the transformation of the outcomes from qualitative research into measurable quantitative values that are used to test the validity of hypotheses. The advantage of this research procedure is that it is a comprehensive holistic testing methodology that is designed to be static way of holistically measuring categorical variables directly applicable to educational and social behavioral environments where the established methods of pure experimental designs are easily violated. The unchanging base of the Tri–Squared Test is the 3 × 3 Table based on Trichotomous Categorical Variables and Trichotomous Outcome Variables. The emphasis the three distinctive variables provide a thorough rigorous robustness to the test that yields enough outcomes to determine if differences truly exist in the environment in which the research takes place ( Osler, 2012 ).
Factor Analysis: A statistical factorial model used in this study is derived from Agbetsiafa (2010) , and Field (2009) . Concisely, factor analysis allows the delineation of an essential or hidden configuration in a data set. It accelerates the analysis of the configuration of the associations (correlation) among an outsized number of variables by describing a set of shared essential measurements, commonly termed factors ( Agbetsiafa, 2010 ).