CDSS Architecture: Oriented on Hierarchical Reinforcement Learning by Automated Planning

Introduction

System analysis is a necessary introductory link to the modeling of objects of arbitrary complexity level, because system errors nullify the efforts spent at the next stages of analysis and/or synthesis of a complex system. As a textbook example, we can cite Ukrainian medical information systems of the first generation, such as Doctor Eleks, Medikit, Helsi.me or Medstar (e.g., Ventskivska et al., 2015): databases are designed with a focus on the medical service (hospital, polyclinic, pharmacy, etc.), and not on the patient, based on the fact that the medical service will pay for the created software product. And the services bought these products, and after paying and filling the databases, they rightly decided that the collected information is their property, and as a result, the patient, turning to different institutions, cannot get a complete picture of the state of his body: different doctors evaluate different data in different standards of their presentation and, quite possibly, do not take into account presumably interrelated symptoms. Obviously, the system analysis of the service object was not carried out, that is, the software company proactively developed software for a potential customer, therefore, from its point of view, some abstract medical service acted as the service object, instead, if the customer was the medical service itself, in its technical task such an object would not be the doctor, but the patient. Only in this case, the data about the patient accompany the patient regardless of the doctor or services to which he applies.

Therefore in data-driven clinical decision support systems (CDSS) it is necessary to make a transition from “discrete” architecture, which involves the separation of patient data, knowledge about treatment methods, heuristic decision-making algorithms, and the interface with the user of the system presented, for example, by Kate Boone (see Figure 1), to a holistic stochastic agent model of the patient in his hierarchical recursive interaction with the environment. The fundamental difference of this approach lies in the deep integration of data into the agent model and its transformation into knowledge. This, in addition to ensuring logical integrity (consistency), makes it possible to use machine learning methods to determine the patterns of empirically discovered dependencies between individual model parameters (data) and explicitly in the form of properties (if the dependencies are unconditional) or rules (if the detected dependencies are accompanied by certain conditions) to include them in knowledge base.

Figure 1.

A “discrete” model of the architecture of a medical decision support system (Taken from Keith Boone) (https://www.youtube.com/watch?v=_1ub86XvuAc)

The core of the system should be an agent model of the organism, which will contain both a universal, common structure and content for some significant group of real persons, the “skeletal” part of the knowledge base, and specific components that will characterize the functioning of each specific patient or their characteristic subgroups, clustered on certain essential characteristics, such as gender, age, race or disease. As a disease is diagnosed, the entire necessary volume of knowledge about this disease is integrated into a specific individual or group, i.e., specific for persons with this disease, agent model of the organism, becoming its integral part in the future. The process of such integration is accompanied by checking and maintaining the logical integrity of the model. Correspondingly, the interdependencies between the parameters of the model, which were affected by adding new data to it, change. Thus, the model is an adaptive knowledge base capable of responding to queries regarding the current state of the organism or being used to predict further functioning, including reactions to medical or other external influences.