Supporting Collaborative Multi-Criteria Evaluation: The VIP Analysis Plug-In for Decision Deck

Introduction

VIP Analysis (Dias & Clímaco, 2000) is a multi-criteria decision support tool to evaluate a discrete set of alternatives in choice problems. Based on the additive model for the aggregation of value functions (Keeney & Raiffa, 1976), its main characteristic is that it does not require a Decision Maker (DM) to indicate precise values for the trade-offs between different criteria. Rather, it can accept imprecise / partial information (namely intervals and linear constraints) on these values. VIP Analysis may be used to discover robust conclusions, i.e. those that hold for every accepted combination of the parameters, and to identify which results are more affected by the imprecision in the parameter values, i.e. the variability of the results.

In its original form, VIP Analysis can be a useful tool for a single user, which can be a single DM, or someone using the tool to support the discussion of a group gathered at the same location, as in a decision conference (Phillips & Phillips, 1993). In this work, we introduce a new version that allows a group of actors engaged in a decision process to collaborate over the internet, hence not needing to meet at the same location, nor at the same time. We have used the Decision Deck (D²) platform (Bisdorff et al., 2007; Decision Deck, 2008a), an open-source software offering a generic multi-criteria evaluation structure where new methods can be plugged-in. The platform allows for workflow-like organization of the decision process, where multiple users can participate in coordinating roles, specialist evaluation roles, or as decision analysts dealing with the more political aspects of the problem (e.g., defining the importance of the evaluation criteria).

Acording to Bergoggi (2003), Internet Multiattribute Group Decision Making (IMGDM) is a new approach to distribute group decision making intending to arrive at a consensus preference ranking of all the alternatives, by aggregating the group members preferences. However, not only DMs may find it hard to provide precise figures about their preferences, but also these preferences may change as the decision making process evolves. The procedures used to elicit the values of the trade-offs (weights, or scaling constants) may require more time and patience than some DMs can spare. Moreover, in group decision situations the opinions and preferences of the DMs diverge frequently.

We argue that the use of incomplete information regarding the trade-offs required by VIP Analysis helps to overcome this situation. As it is emphasized by Sen (1999), “preference formation through social interaction is a major subject of interest […]. It is also important to recognize that agreed social arrangements and adequate public policies do not require that there be a unique “social ordering” that completely ranks all the alternative social possibilities. Partial agreements still separate out acceptable options and a workable solution can be based on the contingent acceptance of particular provisions, without demanding complete social unanimity […]”.

VIP Analysis is based on a similar point of view. The requirement of complete information on the trade-offs between criteria is relaxed, allowing one to proceed with variable interdependent parameters. This means that instead of requiring precise values for the weights it requires just intervals, and/or other linear constraints on these values. The constraints usually stem from imprecise answers from the DM or from holistic judgments about the alternatives that the DM is able to compare.

So, the DMs just fix those constraints on the scaling constants they feel confident to do. Consequently, either in individual or in group settings, VIP Analysis is a constructive approach looking for robust conclusions, trying to identify the “most satisfactory” solution rather than a preference ranking of all the alternatives.

Another key and controversial issue is the discussion on the effectiveness of IMGDM in comparison with the traditional face to face settings.

Communication support for the aggregation of preferences from different DMs must play a central role in IMGDM (Tung & Turban, 1999). It must be taken into account that ambiguity of computer-mediated communication together with the dynamics of the process may lead to hinder the possibility of obtaining good group consensus. Dialogue requirements should be adequate for building effective asynchronous communication settings.