This chapter offers best practices, methods, and strategies for evaluating and assessing coaching services once they have been implemented. In order to determine the extent to which the coaching services that have been implemented are impacting retention, a comprehensive assessment combined with thoughtful analysis of the assessment data must be undertaken on a regular and continuous cycle. The Kirkpatrick and Kirkpatrick (2006) four-level assessment model to evaluate the impact and effectiveness of the selected coaching program on an annual basis, for either an outsourced coaching service or an internal coaching unit or department, is the recommended approach detailed in this chapter. The four levels of the assessment are as follows: Level 1 of the assessment will measure student reactions to the coaching services; Level 2 will assess student learning through the use of pre- and post-coaching assessments; Level 3 will assess transfer of knowledge and skills; and Level 4 will assess the impact and results as a result of the coaching program. The chapter provides advice and discussion about when to conduct each level of the four-part assessment model and a comprehensive sample assessment that can be modified to fit the needs of a wide variety of programs and institutions.
TopIntroduction
This chapter offers best practices, methods and strategies for evaluating and assessing coaching services once they have been implemented. In order to determine the extent to which the coaching services that have been implemented are impacting retention, a comprehensive assessment combined with thoughtful analysis of the assessment data must be undertaken on a regular and continuous cycle. The Kirkpatrick and Kirkpatrick (2006) four-level assessment model to evaluate the impact and effectiveness of the selected coaching program on an annual basis, for either an outsourced coaching service or an internal coaching unit or department, is the recommended approach that is detailed throughout this Chapter. Methodology and approaches to comparing the impact of coaching services and other program and institution wide retention projects are also discussed.
Assessment and evaluation is an important step in the implementation process (Clark & Estes, 2008) that is often overlooked or thought of as a separate project that can be formulated and executed later. However, from my own experience, I believe that it is critical to at least have a basic assessment and evaluation plan as part of your coaching program implementation strategy and timeline. It is also important to recognize that you will need to collect pre-coaching data for all of your students. This pre-coaching or baseline data will be critical to determining the impact of your coaching program and to understanding how successful your coaching program is in meeting your goals or retention benchmarks. Further, it is vital that the outside coaching vendor or the in-house coaching department clearly understand the performance requirements for renewing the contract or continuing as a coaching department along with clearly understanding how and when they will be assessed. Finally, once your assessment levels are complete it is important to timely and transparently share and discuss the assessment data so that all parties to the coaching contract or all stakeholders in the coaching department have an up to date understanding of how the coaching services are being received. Timely sharing of data also provides opportunities to fine tune or revise coaching concepts or elements that are not seen as impactful and successful by the students or that the data identifies as underperforming.
In order to determine the extent to which the coaching services that have been implemented are impacting retention, a comprehensive assessment combined with thoughtful analysis of the assessment data must be undertaken on a regular and continuous cycle. Using the Kirkpatrick and Kirkpatrick (2006) four-level assessment model to evaluate the impact and effectiveness of the coaching program is the recommended approach of this chapter. A recommended timetable of when to deploy each element of the four-part assessment process is included below. Also included below are a sample set of assessment questions that can be modified to fit the needs of a wide variety of coaching programs and institutions as you work through the four levels of the assessment process.
Level One: Measuring Reactions to Coaching
The first level of the Kirkpatrick and Kirkpatrick (2006) model measures participants’ reactions to the coaching services along with providing an initial measure of the motivation levels of the students. The Level One assessment data will also provide insight into what the student has learned from the coaching sessions. In addition, it assesses the extent to which this learning will improve course and program persistence. The recommendation for assessing Level One reactions to the coaching program using the Kirkpatrick and Kirkpatrick (2006) methodology is to utilize or adapt the sample survey described in Table Seven below. The sample survey includes nine Likert scale questions and six open ended short answer questions. It is highly recommended that programs and institutions provide a minimum of three consecutive terms of coaching to all new, readmitted or returning students; therefore, the Level One student survey of reactions to coaching should be launched at the end of the student’s third term of coaching.