Expect What You Inspect: A Worked Example of Dashboards That Support Continuous Quality Improvement in Medical Education

Introduction

The Liaison Committee on Medical Education (LCME) requires that medical schools in the United States and Canada track compliance and continuous quality improvement (CQI) efforts across a broad range of LCME standards, as specified in Element 1.1 of the standards (LCME, 2017). However, the LCME does not specifically state what form these tracking efforts should take, or how medical schools should represent this information to the Committee or internally. The Keck School of Medicine of the University of Southern California (KSOM) has designed a new CQI tracking tool to address LCME’s directive. Built in a commonly available online platform, the KSOM tracking platform serves as a starting template for use and adaptation by other schools and institutions who strive to monitor their ongoing efforts. This worked example provides an overview of the system developed at KSOM, the rationale and theoretical framework for the design, and lessons learned from the development of the system in KSOM’s local context. This chapter presents the results of the development of the CQI tracking platform that began in October of 2017 and concluded in November of 2017, when administrators presented the dashboard during the LCME accreditation visit. Participants in the design and development processes included the Vice Dean for Medical Education, the Director of Educational Technology, two learning scientists, the CQI accreditation administrator, and the support of a medical illustrator. Total development time amounted to approximately 50 hours for all parties over the course of that month.

The platform described in this case was developed to serve as a central location for the multiple activities and parties that surround compliance and CQI tracking. KSOM’s Vice Dean for Medical Education initiated the request for better CQI tracking tools during the school’s Fall 2017 LCME accreditation cycle. As the Vice Dean noted in interviews following the accreditation process:

For this reason, many schools of medicine have turned to the use of ‘data dashboards’ or data display systems to represent their CQI efforts in easy-to-consume formats. While information and analytics representation systems have existed for decades in business and information technology, recent advances in computer and network technologies have made it easier than ever to implement these systems in the context of academic medicine.

In developing this CQI tracking platform, the authors observed that few resources exist to guide the design of compliance and CQI tracking systems for LCME standards and elements. Searches in peer-reviewed literature found few contemporary studies that provide administrators with details about how to create tracking systems that conform to LCME's compliance and CQI tracking expectations. This constitutes a problem with severe consequences, as-without consistent guidelines to produce tools to serve that purpose, medical schools may fail to develop these systems and fail to fulfill the expectations of the LCME standards and suffer consequences such as probation (e.g., Miller, Dzowonek, McGuffin & Shapiro, 2014). Further, a disparity may emerge between large and well-resourced institutions that can invest resources in developing CQI tracking tools and smaller schools of medicine that cannot invest human resource time in the prolonged development of such a system. Finally, these systems can also support the tracking of CQI initiatives beyond the requirements of LCME, thus improving outcomes for endeavors beyond accreditation. KSOM would like to share its response to the challenge of creating CQI tracking systems to help other institutions avoid the pitfalls associated with this issue, along with a theory-driven approach to dashboard development. The components presented in this chapter may help to speed other schools’ advances in CQI tracking and promote an ongoing dialog about how best to achieve the creation of a standardized yet flexible approach to developing these platforms.