Abstract
The accuracy of “stopping rules” for determining the number of clusters in a data set is examined as a function of the underlying clustering algorithm being used. Using a Monte Carlo study, various stopping rules, used in conjunction with six clustering algorithms, are compared to determine which rule/algorithm combinations best recover the true number of clusters. The rules and algorithms are tested using disparately sized, artificially generated data sets that contained multiple numbers and levels of clusters, variables, noise, outliers, and elongated and unequally sized clusters. The results indicate that stopping rule accuracy depends on the underlying clustering algorithm being used. The cubic clustering criterion (CCC), when used in conjunction with mixture models or Ward’s method, recovers the true number of clusters more accurately than other rules and algorithms. However, the CCC was more likely than other stopping rules to report more clusters than are actually present. Implications are discussed.
TopLiterature Review
The problem of determining the number of clusters in a data set is relevant across a wide variety of disciplines such as business, psychology, sociology, statistics, biology, engineering, and computer science (Li et al., 2008; Liao & Ng, 2009; Milligan & Cooper, 1985; Wedel & Kamakura, 2000). The following review briefly highlights many of these techniques and while not exhaustive, is designed to provide a sense of the various approaches that have been proposed (Carmone, Kara, & Maxwell, 1999; Dolnicar & Leisch, 2010; Li et al., 2008; Milligan & Cooper, 1985; Milligan & Hirtle, 2003; Maulik & Bandyopadhyay, 2002; Salem & Nandi, 2009).