Abstract
The Liaison Committee on Medical Education (LCME) requires that medical schools track compliance and continuous quality improvement (CQI) efforts across a broad range of LCME standards. However, LCME does not state what form these tracking efforts should take, or how medical schools should represent this information to the Committee or internally. This chapter provides an overview of the Keck School of Medicine of the University of Southern California's (KSOM) new approach to CQI tracking using an online dashboard. The project resulted in an online platform that represents the CQI project progress across a range of elements, maintains visual consistency across a range of data sources and file types, and is easily accessible by relevant stakeholders. This innovation from KSOM illustrates how a web-based platform supports CQI efforts, and how this design can be translated to other contexts. The design presented in this chapter provides guidelines for the development and innovation of CQI tracking initiatives at other schools.
TopIntroduction
The Liaison Committee on Medical Education (LCME) requires that medical schools in the United States and Canada track compliance and continuous quality improvement (CQI) efforts across a broad range of LCME standards, as specified in Element 1.1 of the standards (LCME, 2017). However, the LCME does not specifically state what form these tracking efforts should take, or how medical schools should represent this information to the Committee or internally. The Keck School of Medicine of the University of Southern California (KSOM) has designed a new CQI tracking tool to address LCME’s directive. Built in a commonly available online platform, the KSOM tracking platform serves as a starting template for use and adaptation by other schools and institutions who strive to monitor their ongoing efforts. This worked example provides an overview of the system developed at KSOM, the rationale and theoretical framework for the design, and lessons learned from the development of the system in KSOM’s local context. This chapter presents the results of the development of the CQI tracking platform that began in October of 2017 and concluded in November of 2017, when administrators presented the dashboard during the LCME accreditation visit. Participants in the design and development processes included the Vice Dean for Medical Education, the Director of Educational Technology, two learning scientists, the CQI accreditation administrator, and the support of a medical illustrator. Total development time amounted to approximately 50 hours for all parties over the course of that month.
The platform described in this case was developed to serve as a central location for the multiple activities and parties that surround compliance and CQI tracking. KSOM’s Vice Dean for Medical Education initiated the request for better CQI tracking tools during the school’s Fall 2017 LCME accreditation cycle. As the Vice Dean noted in interviews following the accreditation process:
Our formal tracking of CQI initiatives began with the LCME process. Prior to that, there was not a central mechanism to track [improvement initiatives] from the various areas across the medical school. We maintained this data, but it was in the form of program evaluation data, and it was located in various offices. The information was brought together from those various sources and reviewed it at the appropriate meetings and gatherings, so it was not centralized. And as we learned going through the LCME process, we needed to capture all the relevant information and bring it together in one location. (2018)
For this reason, many schools of medicine have turned to the use of ‘data dashboards’ or data display systems to represent their CQI efforts in easy-to-consume formats. While information and analytics representation systems have existed for decades in business and information technology, recent advances in computer and network technologies have made it easier than ever to implement these systems in the context of academic medicine.
In developing this CQI tracking platform, the authors observed that few resources exist to guide the design of compliance and CQI tracking systems for LCME standards and elements. Searches in peer-reviewed literature found few contemporary studies that provide administrators with details about how to create tracking systems that conform to LCME's compliance and CQI tracking expectations. This constitutes a problem with severe consequences, as-without consistent guidelines to produce tools to serve that purpose, medical schools may fail to develop these systems and fail to fulfill the expectations of the LCME standards and suffer consequences such as probation (e.g., Miller, Dzowonek, McGuffin & Shapiro, 2014). Further, a disparity may emerge between large and well-resourced institutions that can invest resources in developing CQI tracking tools and smaller schools of medicine that cannot invest human resource time in the prolonged development of such a system. Finally, these systems can also support the tracking of CQI initiatives beyond the requirements of LCME, thus improving outcomes for endeavors beyond accreditation. KSOM would like to share its response to the challenge of creating CQI tracking systems to help other institutions avoid the pitfalls associated with this issue, along with a theory-driven approach to dashboard development. The components presented in this chapter may help to speed other schools’ advances in CQI tracking and promote an ongoing dialog about how best to achieve the creation of a standardized yet flexible approach to developing these platforms.
Key Terms in this Chapter
Data Visualization: A visual representation of data that uses a range of graphic forms to help the user make sense of a particular set of data.
Artificial Intelligence (AI): In the context of this chapter, a term for analytic tools that use advanced algorithms to analyze large data sets and augment human analysis of data.
Data Dashboard: A grouping of related data visualizations that provides users with information about the state of a system.
Continuous Quality Improvement (CQI): A business process philosophy that promotes goalsetting, data collection, and program evaluation towards the end of providing students with better experiences and outcomes.
Big Data: A term used to describe large-scale data sets that include multiple kinds of demographic, behavioral, or performance data. Sometimes used to describe the unique kinds of data analysis that can be performed on these data sets.
Data Warehouse: A data system that ingests, stores, and retrieves data for use in other systems.
Key performance indicator (KPI): Proximate measures of performance that are used to identify how well a given program or system is operating based on predefined goals.
Liaison Committee on Medical Education (LCME): A joint body of the American Association of Medical Colleges (AAMC) and the American Medical Association (AMA) and other groups that evaluates and accredits physician training facilities in the United States and Canada.