Measuring Disagreement

Abstract

Agreement is an important goal of computer-mediated and face-to-face groups. This paper suggests a measure disagreement in groups facing limited choices, as in a multi-choice questionnaire. It defines a disagreement score between two people, then takes one person’s disagreement as the average of their pair-wise scores with the rest of the group, and finally defines the group disagreement as the average of its member’s disagreement. This gives a standard disagreement scale (from 0 to 1) for any group response pattern, for any size group, facing any number of choices. It can be inverted to give agreement, though this does not necessarily predict group coalescence. It is encouraging that when the method is extended to ranked, interval or ratio scale data, it is equivalent to the score variance, and that it also matches an ecological diversity measure. Unlike variance, this measure can be used with categories, and gives both individual and group values. Being standard, it offers a single score in cases where the group size and number of choices faced is unknown, e.g., online computer-based group feedback. Examples are given of how the measure can be used.