A Comparison of Pairs, Triads and Quads in Multi-Attribute Decision Making

Introduction

Individual decision-makers and organisational groups are often asked to evaluate large sets of items as part of their routine managerial responsibilities. Such multi-attribute evaluations span various contexts: administrators evaluating diverse budget requests (Shen, Lo & Wang, 1998), steering committees prioritising multiple competing project proposals (Jessop, 2002), individual product designers deciding on which product qualities to focus (Horowitz & Zappe, 1995), policy makers considering options for urban waste management (Bollinger & Pictet, 2008) or paper recycling (Pati, Vrat, & Kumar, 2008), and many others. Although evaluating a large set of attributes is a common managerial task, it’s not easy to execute effectively (Zeleny, 2008). Regardless of the approach used, decision makers must implicitly or explicitly consider their strength of preference for one attribute over another (Lai, 2001). Various innovative techniques have been developed to elicit the weights of attributes as judged by decision makers, using subjective, objective and integrated approaches (Wang & Luo, 2010; Wang & Parkan, 2006).

One prevalent multi-attribute evaluation method that is used to carry out such evaluations is based on pair-wise comparisons (Lahdelma, Salminen, & Kuula, 2003). This method decomposes the evaluation task into a series of judgment-based choices that consider a pair of attributes at a time and then utilizes statistical techniques to infer the implicit importance weights from the decision-maker’s choices (Schoemaker & Waid, 1982). Research suggests that pair-wise comparisons and other methods that decompose the evaluation task into a series of choices may be preferred over holistic methods (such as point allocations and simple rankings). This may be because comparison-based evaluations of large sets have lower cognitive load requirements (in comparison to holistic techniques that require the simultaneous consideration of all attributes) and thus avoid some of the reliability problems associated with cognitive overload (Srivastava, Connolly & Beach, 1995).

While the pair-wise technique has become the defacto standard in comparison-based evaluations and is quite useful in reducing task complexity, it suffers from two major limitations when used to assess a large number of attributes (Lahdelma et al., 2003). First, the time that is required to complete the evaluation of the pairs can be quite lengthy. When employing the pair-wise comparison approach, each pair of attributes must be evaluated to assess the relative importance of one attribute to the other. For n attributes, this results in comparisons. For example, an evaluation of 25 attributes would require at least 300 pair-wise comparisons, a highly time-consuming task for most decision-makers. Second, the large number of independent comparisons in pair-wise judgments may result in conflicting choices and lack of transitivity (Flynn, Sakakibara, Schroeder, Bates & Flynn, 1990). To illustrate, assume that a respondent is asked to evaluate three attributes (A, B and C) using pair-wise comparisons. If the respondent were to rank A > B, B > C, and C > A, an intransitivity occurs (since the first two comparisons would imply A > C).