A Multi-Criteria Decision-Making Model for Selecting a Maturity Model

This paper proposes a decision model for a maturity model choice problem via the multiple-criteria decision analysis method. The authors implemented the model in FITradeoff decision support system to select a project management maturity model for a Brazilian industry operating in the sector of distributing natural gas. FITradeoff is a flexible and interactive procedure of elicitation for multi-criteria additive models that requires only partial information from the decision maker (i.e., there is no need to elicit very detailed information from the decision maker, an approach that the decision maker can find laborious and tiring). The authors observed that the use of a multi-criteria approach imposes certain rigor and pattern on the decision process to select a maturity model in project management. Applying the model enabled comparison of information from the four maturity models and therefore selecting a project management maturity model based on the decision-maker preferences.


INTRoDUCTIoN
When an organization needs to decide which domain to improve, many different directions may be taken.There are a number of different changes that are possible since, in many cases, it is unclear exactly how to modify a process (Helgesson et al., 2012).That is, is not easy to decide how to change a process in the best way possible.In order to ease the decision-making process, maturity models (MMs) are used.MMs accept that expected patterns (theorised in stages) exist in the development of organizations and aid organizations to direct their improvement plans (Gottschalk & Solli-Saether, 2006).
Studies have reported that, in some cases, there is more than one model for the same domain (Wendler, 2012;Santos-Neto & Costa, 2019).The large number of models for the same domain may hinder the decision by which MM best aligns with organizational objectives.One way to solve this problem is to use

BACKGRoUND
The essence of MMs lies in their evolutionary nature, namely, they consist of a series of stages in which the complexity of the business activity is increased from one level to another until the last stage, at which time the highly complex activity is managed perfectly (Serna, 2012).Furthermore, such models strengthen businesses by ensuring that they have set up the operational conditions needed to manage organizational change satisfactorily.
Although the MM approach was first used in computing and software engineering, its application has spread to various other fields (Santos-Neto & Costa 2019), such as supply chain management (McCormack, 2008), business process management (Lima et al., 2017), education (Bollin et al., 2018), agile process management (Nurdiani et al., 2019;Gren et al., 2015), and, as in this paper, PM (Neves et al., 2013;Crawford, 2006).PM is one of the approaches that stands out most in the MM literature.This topic is attracting increasing attention in business and academic circles by offering practices that aim to contribute to approaching organizational problems in a systematic way (Scotelano et al., 2017).
Apart from needing to have a well-defined beginning and end, a project involves a series of steps with specific objectives that require monetary, human, and material resources to be available in the amounts and at the level of knowledge and experience needed to meet tight deadlines (Neves et al., 2013).
Maturity Models for the PM domain have an important role in helping organizations make decisions and adopt best practices in PM.Project Management Maturity Models (PMMMs) are designed to assess the ability of an organization to make the best use of resources that are aligned to the best practices for the domain in question.Thus, maturity in PM seeks to express the extent to which an organization is able to implement its projects and achieve its goals (Andersen & Jessen, 2003).That is, organizations with high maturity level in project management domain avoids the problems proactively by predicting them before they occur (Cerdeiral & Santos, 2019).
According to Görög (2016), PM maturity is a higher occurrence topic in both international conferences and professional journals.Several organizations are developing MMs in PM.However, in this study, was selected only four MMs quoted in the literature for the choice problem.In order to make our selection, was consulted a systematic review performed by (Santos-Neto & Costa, 2019) in which the objective was to examine published papers that presented research developments in MMs.
In their study, they identified 15 papers with a focus on MM applications in the PM domain.The PMMMs which were more frequently applied, according (Santos-Neto & Costa, 2019), were PM Solutions' PMMM (Crawford, 2006) and the Organizational PMMM (OPM3) (PMI, 2008).
In addition to PM Solutions' PMMM and OPM3, was selected Kerzner's PMMM (KPMMM) (Kerzner, 2017) and Prado's PMMM (P2M3) (Neves et al., 2013).KPMMM because this model can evaluate maturity levels throughout the various phases of an organization's PM life cycle (i.e., its embryonic, executive management acceptance, growth, and maturity phases) (Bay & Skitmore, 2006).And, P2M3 was selected because it is in tune with Brazilian culture, the country in which the company which is the focus of this study is located (Neves et al., 2013).
Table 1 compares the PMMMs selected for this study.It indicates the maturity level of each model, whether or not elements of the Guide to the Project Management Body of Knowledge (PMBOK guide) have been incorporated into each model's structure and the main advantages of each model.
PM Solutions' PMMM is considered to be a formal tool and has the Capability Maturity Model (CMM®) as its theoretical basis.It has five evolutionary levels of maturity, which are aligned to developing maturity in the nine areas of the PMBOK guide -Guide to the Project Management Body of Knowledge (PMI®) (Appleby, 2007).It has evaluation software that is made available to PM Solutions' official consultants for individual evaluation only.The OPM3 is the maturity model developed by the PMI Institute, which publishes the globally accepted standard for project management, the PMBOK (Kwak et al., 2015).OPM3 presents itself as possibly the most powerful model in the world market.Upon analysing its internal characteristics, the model can be considered robust but complex since it has an evaluation instrument with more than 150 questions, a database with approximately 600 best practices, and provides an internal program to assist in developing the improvement plan (PMI, 2008).KPMMM was developed in 1998 and is aligned with the practices of the PMBOK guide (Bay & Skitmore, 2006).KPMMM is applied by using a questionnaire containing 183 questions, which are split into sub-sections having 80, 20, 42, 25 and 16 questions, respectively, for each of the five maturity levels present in the model (Kerzner, 2017).The P2M3 is characterized by the simplicity of its questionnaire, its practical way of obtaining the scores, and its applicability to the various sectors of an organization, as well as to the organization as a whole (Souza & Gomes, 2015).As mentioned previously, P2M3 is in tune with Brazilian culture since it has been used by many Brazilians institutions.Additionally, it is available online at 'http://www.maturityresearch.com(Neves et al., 2013).The dimensions addressed in the model are: technical and contextual competence, practical use of methodology, computerization, organizational structure, alignment with the organization's business (strategic alignment), and behavioural competence.

THe FITRADeoFF MeTHoD
FITradeoff is a multi-criteria method built on a DSS that uses a notion of flexible elicitation; in other words, it requires less effort from the DM (Almeida et al., 2016).Just as in a traditional trade-off (Keeney & Raiffa, 1976) and SWING (Mustajoki et al., 2005), FITradeoff elicits weights in a process that allows the alternative with the highest added value be identified.Applying the DSS defines the weights of Multi-attribute Value Theory (MAVT), in which the alternatives are scored according to the function value (Eq. 1) of additive aggregation (Lima et al., 2017;Santos et al., 2022): In this equation, v(a j ) is the global value of alternative j, k i is the weight of criterion i, and v i (x ij ) characterizes the value of alternative j for criterion i.
The flexibility of the FITradeoff means that the elicitation procedure can be changed easily and adapted to different conditions, periods and circumstances.Thus, the path upon which the procedure is implemented may not follow all the steps that the standard procedure requires.Instead, the steps are chosen according to the different conditions that arise during the process (Almeida et al., 2016).Hence, the expectation is that the DM will offer less information than in the standard procedure.
In the traditional tradeoff procedure, an indifference (I) relationship must be obtained between the two consequences to find the value of the scale constant ki.Therefore, the traditional tradeoff based method requires complete DM information to find the solution to the problem (Lacerda, Santos-Neto & Martins, 2021).The comparison process is repeated, providing information until a solution can be reached.
The DSS for the FITradeoff application is available free of charge at 'http://fitradeoff.org/'.The method follows three steps during which the flexible and interactive elicitation process can be applied (Almeida et al., 2016).Figure 1 shows the procedures for applying FITradeoff.
In step 1, the value function is obtained for each alternative and each criterion.In this case, it should be ensured that the appropriate normalization procedure is applied since the elicitation procedure in an additive model that depends on the scale adopted (Frej et al., 2017).Although there are other normalization methods, the DSS automatically normalizes the data as shown in (Eq.2) (Lima et al., 2017): where v´j(a i ) is the normalized value, and v j (a i ) is the value of alternative j for criterion i.
Step 2 is divided into 2.1, in which the criteria are ordered, and 2.2, where the attempt to solve the weight space selection problem occurs using the information obtained in 2.1.In 2.2, LP is used to solve the problem of the objective function (Eq. 3) and determine what the restrictions are for (Eq.3) and (Eq.4) (Lima et al., 2017).
Subject to restrictions: where k i represents the weight for criterion i, v i (x ij ) is the value of alternative j for criterion i, and v i (x iz ) is the value of alternative z for criterion i.
If a single solution is found, the process is finalized.During finalization, the weight ranges that support the solution are calculated and produced in a report which makes the final recommendation (Almeida et al., 2016).If a single solution is not found, step 3 is initiated.At this point in the FITradeoff, a compensation procedure is executed to find the DM's preferences.The consequences are explored, and sequential tries are made to attempt to resolve the problem with the information obtained to that instant.The third step begins by requesting the DM to choose between two consequences: the first consequence, in which all the criteria, with the exclusion of criterion 1, are assessed with the worst outcomes, and second consequence, where all the criteria, with the exception of the last criterion, are assessed with worse outcome.The comparison procedure is repeated (for other consequences), and, concluded each DM answer conferring to his/her preferences, new restrictions will be used in an LP model (Lima et al., 2017).The LP model orders the alternatives while taking into account the restrictions that have been incorporated into the model as preferences are found.Doing so allows the alternatives to be classified into three different conditions: potentially optimized, dominated, or ideal (Lima et al., 2017).The step is finished if a unique solution is found.Further detail on the procedure can be found in Almeida et al. (2016).

DeCISIoN MoDeL PRoPoSeD
As discussed in the Introduction, this study seeks to select a MM to PM using the DSS FITradeoff.The problem will be dealt with by using a multi-criteria approach.Specifically, modelling will be applied in order to choose an MM in PM, taking into account the DM's preferences and the evaluation of a set of alternatives using multiple criteria.The FITradeoff is a multi-criteria method with a compensatory rationality.In a decision problem, a compensatory, or a non-compensatory, rationality should be checked with the DM's preference before choice a multi-criteria method (Martins et al., 2017).When asked to DM, a compensatory rationality was chosen for this application.
Was created a research model based on the framework defined in Almeida et al. (2015).The flow-chart of the decision model proposed is given in Figure 2.
In the preliminary phase, the first step of the research model consists of characterizing the DM and other actors.At this step, the person responsible for deciding which PMMM to adopt for the organization must be identified within the organization.Other actors may also contribute to the problem.For instance, specialists may support the DM by providing information for the decision process (Almeida et al., 2015).
The second step is performed to structure the problem of choice.In this step it is defined the family of criteria and the set of potential MMs.The criteria should represent the objectives of the problem decision must be identified and always be validated with the DM (Almeida et al., 2015).Additionally, in step two, the set of decision alternatives identified must be able to present a PMMM that could reach the objective of the problem.
For step three, construction of the decision matrix, in the preferences modelling phase, the MMs identified for the domain approached in the previous steps should be evaluated according to each criterion defining and building the consequence matrix: PMMM versus criteria.For this step, the literature is consulted in order to gather information to evaluate the performance of each alternative in each criterion of the problem.
In step four, the multi-criteria FITradeoff method should be applied according to the procedures described in Section 3 of this paper, and, the results of the models are analysed in step five.At this step, a sensitivity analysis is performed through the DSS.Thus, variations in the parameters used can be executed by building situations to confirm the robustness of the resultand.
In step six, the suggested solution is presented to the DM.In the following section, the proposed model is applied and the steps used are discussed in greater detail.

APPLICATIoN oF A DeCISIoN MoDeL
The company focus of this study has a concession to work with projects in the following services: the implementation of technological research services, exploration, production, acquisition, and storage of natural gas; the independent production and sale of electric energy; the transport, import, export, manufacture, and assembly of the components needed to supply the gas market; and the distribution and sale of gas and/ or by-products and derivatives.The manager of the project office organization, who has worked for the company for more than 14 years and has undertaken his current function for more than 7 years, was designated as the DM, according to step 1.
In the second step, which is to define the family of criteria, it was taken into account that each criterion is seen as representing objectives, i.e., the criteria must be defined in order to measure the performance achieved in meeting the objectives (Almeida et al., 2015).The objective of the problem was defined as selecting a PMMM that can be applied to any segment that reliably expresses the real maturity in the PM of the organization, that allows the creation of an evolutionary action plan, and that is not exhausting to apply.The family of criteria was built from research in the MM evaluation literature.The criteria were extracted from three surveys that were conducted for the studies by Lima et al. (2017), Van Looy et al. (2013), and Man (2007).
The first two of these studies presented criteria for evaluating MMs for managing business processes (BPMMs), and Man (2007) suggested evaluation criteria for an MM in PM.Were selected validated criteria that could be collected from the data available and that were accessible in the literature.Moreover, was collected criteria that could be used with respect to choosing MMs applicable to any domain.Table 2 details the family of criteria, which consists of 12 criteria, as well as the scale of evaluation for each criterion, a brief description of each criterion, and the source of each criterion.For further clarification on the criteria selected, see the studies by Lima et al. (2017), Van Looy et al. (2013), andMan (2007).
As to the set of alternatives, was opted to choose four MMs in PM with great visibility in the literature that could reach the objective of the problem.As detailed previously, the MMs chosen to comprise the set of alternatives were: PM Solutions' PMMM (Crawford, 2006), OPM3 (PMI, 2008), KPMMM (Kerzner, 2017), and P2M3 (Souza & Gomes, 2015).
For the third step, the construction of the decision matrix, the criteria were evaluated based on information contained in the literature.To identify the validation method, the following studies were  costs, and the number of assessment items.While these criteria were given a weight of 0.14, the others were weighted with a value of 0.07.The maximum value found, 0.82, results from multiplying the normalized value function of P2M3 in each one of the criteria by the respective weights obtained from the results of the FITradeoff method.
For a sensitivity analysis to be performed, for any set of weight vectors within the space defined by the minimum and maximum limits, the global value of the alternative P2M3 is greater than the global value of the other alternatives.Thus, it can be seen that P2M3 is the best alternative for a large range of weight values (Figure 5).This demonstrates that the model can be considered robust for the result found.
Moreover, the DSS enabled a sensitivity analysis to be performed with a variation of the weights.The process was carried out using simulations with a variation of ± 10% in the weights of all the criteria.The result was that in only 3.69% of the simulations was there a change in the model chosen.As to the set of Potentially Optimal Alternatives, P2M3 was allocated in 100% of the simulations, and in 3.93% of those for KPMMM.This demonstrates that the model can be considered robust for the result found.

CoNCLUSIoN
The main contribution of this study is that it shows that the MCDM method can be considered a significant resource in choosing an MM.Specifically, FITradeoff proved to be an efficient method for seeking to achieve this objective in the PM domain.It is important to emphasize that the algorithm behind the DSS allows the DM to provide partial information during the decision process, which makes the procedure less exhausting for him/her and thereby facilitates the use of the application and the decision-making process.
Was undertook this study to develop a decision model for MM selection.To this, was structure a multi-criteria decision model based on the FITradeoff DSS. Figure 2 shows the decision model proposed with six steps.Was performed an application to test our decision model and validate it for the problem of PMMM selection.

Ranges of weights
Through apply the six steps, was observed that the result represents the DM's preference for P2M3 (Table 4).Though, it is important to emphasize that other results can be found in other contexts, e.g., a new ordering of criteria or the inclusion/exclusion of new criteria and/or alternatives.As to the applicability of the method, from the DM's point of view, in this problem, the number of questions needed to find a final solution was just one.This reveals that the FITradeoff did not demand the DM to make much effort to solve this problem.However, variations in how decisions are made tend to coincide with variations in who makes the decision (Yates and Oliveira, 2016).Thus, the number of questions that must be asked under the FITradeoff method to find a unique solution is not, in general, fixed, i.e., it varies in accordance with the DM's preferences and, the data of the particular problem.The DM was asked about the managerial perspective with future MM application in the unit.The DM reported that, in recent years, the organization has directed efforts to improve PM, so a positive assessment was expected with this application.
As a possible limitation, in Table 2 was proposed a family of criteria that was validated based on the opinion of only one DM and only used to assess maturity in the field of PM.Therefore, a gap that can be filled by future studies is to search for a method that validates criteria based on procedures that draw, in depth, on the opinions of a sample of experts with wide experience that differ from each other.
It is worth underlining the importance of this study in terms of the maturation of research related to maturity models.As already presented above, several MMs have already been developed.However, there is little concern about how to direct the application of such models to the DM's goals and preferences.In addition, every day at many organizations, important decisions are made that directly affect their results and the probability of success in their activities.Research that addresses the diverse use of MCDM methods in practice contributes to making available to society alternatives better able to analyse information and make decisions.Thus, it is hoped that this study may provide a means for choosing an MM that is applicable to supporting managers in the PM domain.

ACKNowLeDGMeNT
This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior -Brasil (CAPES) -Finance Code 001, the research programme funded by the Brazilian Research Council (CNPq), the Federal University of Mato Grosso do Sul (UFMS), and Federal University of Pernambuco (UFPE).

Figure 2 .
Figure 2. Decision model for the MM selection problem based on FITradeoff Figure 4.The performances of consequences A and B