Model Adequacy
In most cases, the model is a mathematical description of the processes in a partially or fully completed object. One of the main requirements for modeling is to give correct and understandable data obtained as experimental results. Thus, a model becomes a means of supporting decision-making based on scientific assumptions. The usefulness of a given model is assessed by its sustainability for a specific purpose, and one of the main tasks of computer modeling is the creation of an adequate model that reflects the structure of the modeled system and the processes taking place in it with sufficient accuracy (Parker, 2020). The article seeks answers to three main questions: “What does it mean for a model to be adequate-for-purpose? What makes a model adequate-for-purpose? How does assessing a model’s adequacy-for-purpose differ from assessing its representational accuracy?”
It is generally understood that a model is adequate when it faithfully fulfills its purpose and describes the modeled object with a permissible approximation in a structural, logical and mathematical aspect. In this reason, the adequacy of a given model in the general sense means the correspondence between the model and the modeled object and determines how accurately and completely it reflects the structure and functioning of the original. A reliable assessment of the adequacy of a given model is possible with a correctly defined main objective and formulated criteria for performing the assessment, and it is usually recommended to use a combination of several appropriately selected different techniques. This is especially important when processing large data sets and complex models, where an incorrectly developed model “could mislead researchers and compromise their inference” (Carstens et al., 2022). Three main challenges for researchers are defined: (i) correct choice of modeling method and tools; (ii) correct assessment of analytical results or their interpretation; (iii) evaluation of the model and it’s fit with the studied object. The paper states that a variety of approaches can be used to assess model adequacy, from simple visual inspections to statistical fit assessment tests, with the increased power and interpretability of statistical approaches justifying their increased complexity. In addition, there is a review of existing software packages for testing model adequacy. The question of the suitability of the statistical evaluation of a model is also advocated in (Fonseca et al., 2022), where it is confirmed that “naive analysis of data ... may lead to spurious results”. To overcome such a shortcoming, the paper proposes a package to complement the log data exploration capabilities of Bayesian Skyline analyses. The goal is to simulate predictive data sets to compare with statistics calculated from the empirical data to check for model violations.
It should be noted that the model cannot completely replace the physical object. In the process of designing the model, its adequacy is artificially distorted due to the idealization of several components and influencing factors. This necessitates continuous assessment of adequacy, which can be achieved by comparing with real data (estimates) or results from another type of modeling (or monitoring), recommending the following sequential steps.
The initial assessment of model adequacy should be made already when formulating the conceptual model. It is related to the analysis of the real environment and the correct reflection of its structure and dynamics in the developed model. Such an assessment is too subjective, as it depends on the skill of the developer to reflect the real world in abstract forms and correctly form the questions whose answers will determine the level of adequacy.
The second stage in assessing adequacy is related to the specific implementation of the mathematical model and its programming in a selected language environment. Since the design of a computer model involves several successive steps, each of them affects the overall adequacy through its specific results. In this reason, the assessment of adequacy is based on a comparison with the previous state of the model for each step. For example, mathematical dependencies are compared to the concept of the model, the logical structure to the mathematical description, the descriptive functional algorithm to the logical structure, etc.
The final assessment of the adequacy of the program model can be obtained through test variants prepared independently of the implementation of the model. The coincidence of the results of the control modeling with those of the test variants testifies to the adequacy of the model for a given type of tasks. This is also the task of the model validation stage, which aims to achieve high reliability and correctness of the output results when working with real data. Test cases can be developed based on other types of models or by data on the behavior of a real system. The second approach would produce better results but is more difficult to implement.
To achieve high reliability of the computer model, it is necessary to assess the current adequacy to accompany the sequence of model development.