Abstract
The New Faculty Preparation Survey was designed to measure perceptions of recently hired faculty regarding research, service, and teaching for self and institution, while considering academic preparation. A Rasch measurement model was applied to the calibration sample responses to evaluate the psychometric properties of the instrument, providing a foundation for structural modifications. Rasch results revealed sound reliability; however, item misfit existed, coupled with limited utility of the full rating scale. Studying the measurement properties of the instrument will aid in proper interpretation of the results and in the construction of an instrument that can be used by higher education institutions to better understand faculty, and potentially to enhance programmatic structuring. Accurately measuring and understanding faculty perceptions across research, service, and teaching can aid in professional development training at the university level, future faculty preparation, and general faculty satisfaction.
TopIntroduction
When evaluating the quality of any survey instrument, reliability and validity are critical to the process. “Exemplary manuscripts… use measures that have established psychometric merit, and they provide evidence of the reliability and validity of those measures” (Smart, 2005, p. 470).This lack of consideration of psychometric attributes is problematic in the higher education literature, and even more so, on a daily basis with the collection of information utilized throughout the academic community. Collecting information regarding faculty perceptions related to research, service and teaching could be beneficial, but the necessity to collect data that is credible and has utility must first be considered. This study provides a methodological framework for constructing a quality data collection tool and highlights the potential benefits of the New Faculty Preparation Survey.
The Rasch measurement model was applied to the 1997 New Faculty Preparation Survey calibration sample data to evaluate the structure and utility of the instrument, which was designed to measure perceptions of recently hired faculty regarding research, service and teaching. The study provides a foundation for accurately operationalizing, measuring, and interpreting faculty perceptions across a variable termed ‘faculty load’, which includes research, service, and teaching. As survey research becomes increasingly utilized in higher education decision making, it is essential that researchers set high standards for the process of survey development. Here, a healthy balance between content and measurement considerations has been met through collaboration of interested parties. Finally, this study provides a model where an audience not proficient in item response theory can still remain involved throughout this survey development process.
Key Terms in this Chapter
Faculty Load: The way in which faculty positions distribute efforts in research, teaching, and service responsibilities.
Faculty Perceptions: Responses from faculty measuring affect such as attitudes, beliefs, values, etc.
Calibration Frame: Sampled participant responses used in the model to create the measurement continuum.
Calibration: The method of using the Rasch model to convert estimates into an interval continuum in which all persons and items are comparable along the same scale.
Fit statistics: Measures that are examined with per-established criteria to determine if the data are suitable for the model use.
Rasch Measurement Model: A probabilistic model that utilizes all person responses to all items to create a uni-dimensional scale in which all persons and items have estimates along the same continuum.