Efficient Optimization Formulation Through Variable Reduction for Clustering Algorithms

Efficient Optimization Formulation Through Variable Reduction for Clustering Algorithms

Priyanka Devi Pantula (Indian Institute of Technology Hyderabad, India), Srinivas Soumitri Miriyala (Indian Institute of Technology Hyderabad, India) and Kishalay Mitra (Indian Institute of Technology Hyderabad, India)
DOI: 10.4018/978-1-5225-2990-3.ch006
OnDemand PDF Download:
List Price: $37.50


Clustering based unsupervised learning methods are gaining importance in the field of data analytics, owing to their high accuracy, simple implementation and fast computation when compared with conventional supervised learning methods. Among several types of clustering techniques, those implying optimization routines are found to be more efficient. However, explosion in number of decision variables is making these algorithms computationally intensive. The authors present an efficient two stage optimization problem formulation based fuzzy clustering method which works through variable reduction approach. The membership values associated with each data point, forming the majority of decision variables are estimated as parameterized network outputs similar to ANNs. The reduction in decision variables allows the implementation of evolutionary optimization solvers to solve the multi objective optimization problem for clustering. Further the second stage optimization problem estimates the optimal number of clusters. Results with prominent case studies from literature are presented
Chapter Preview



Data analytics is one fast growing research field owing to the availability of enormous volumes of data from processes across several engineering domains (Hair, Anderson, Babin, & Black, 2010). Say it be the calcium oscillatory signals obtained from the minute nerve cells or the process data obtained from a gigantic blast furnace, scientists are now majorly focusing on identifying a trend in data which would allow them to build data based models and perform studies such as the sensitivity analysis to better understand the system in hand (Sturn, Quackenbush, & Trajanoski, 2002). Lack of physics based models for these complex practical phenomena has further propelled the growth of this field in all domains. Among the ongoing research in data analytics, the scientific community is broadly divided into two groups – one which focuses on supervised learning of the data and the other where unsupervised learning methods are applied for pattern recognition (Last, Szczepaniak, Volkovich, & Kandel, 2006). Models like Artificial Neural Networks (ANNs), Support Vector Machines (SVMs) and so on work primarily on the methods of supervised learning (Kotsiantis, Zaharakis, & Pintelas, 2007). Cluster analysis is one classic technique which works with unsupervised learning. ANNs and SVMs learn from the given data by training the parameters such as the weights and biases. These models are often over-fitted to the data thereby losing their predictability for unseen data (Dietterich, 1995). Cluster analysis on the other hand is primarily a classification technique, where the given data points are segregated into clusters/groups with similar features (Kaufman, & Rousseeuw, 2009). It is proven to be a useful tool in several exploratory pattern recognition analysis, decision-making, data mining, document retrieval and image segmentation (Kaufman et al., 2009 and Vasant, 2016). For example, in the area of marketing research, the companies wish to divide their market into distinct groups based on the consumers with similar needs or wants so that different targeted marketing programs can be developed with the help of this knowledge. However, in many such real-world problems, the information available about the data is less and hence the decision-maker needs to make some assumptions in order to analyze the available data. Under these restrictions, the clustering methodology is particularly appropriate for the exploration of interrelationships among the data points to make a preliminary assessment of their structure.

Whatever be the method of learning implemented in all the supervised and unsupervised techniques, in most cases, they all end up solving a Non-Linear Programming (NLP) problem, which necessitates a robust optimization solver (Jain, 2010). In this chapter, the authors focus on significance of optimization solvers implemented to solve the NLP problem emerging from the unsupervised learning through cluster analysis focusing mainly on fuzzy clustering methods. The authors have selected Fuzzy based clustering methods due to the severe amount of complexity present in NLP problem and extreme difficulty caused by the explosion of decision variable space (Yang, 1993). Many fuzzy clustering algorithms such as FCM, PCM, FPCM and so on are based on the idea of optimizing an objective function. Usually, this objective function depends on the (i) distance between the cluster centers and data points and (ii) weighted membership degrees. By taking the first derivative of the objective function with respect to the cluster parameters, one obtains the necessary conditions for the objective function to have an optimum. These conditions are then applied in an iteration procedure for obtaining the optimum cluster parameters (Nayak, Naik, & Beher, 2015). This requires the objective function to be differentiable and the iteration procedure can be computationally efficient only if the derived conditions lead to explicit equations for the cluster parameters. The set of cluster parameters, that determine the size and the shape of a cluster, depends on the specific application field. In this chapter, main focus is laid on fuzzy clustering which serves as an explorative data analysis method, especially for unsupervised classification tasks, techniques for rule extraction (for instance for fuzzy controllers) (Vasant, 2013) and shell clustering algorithms (Klawonn, Kruse, & Winkler, 2015), that are designed for boundary detection in image recognition. Unfortunately, in many cases the restrictions that are enforced on the cluster parameters to be able to derive the iteration procedure are too narrow and exclude a lot of interesting possibilities. Thus it is desirable to have an alternative optimization technique that allows for more freedom in the choice of the cluster parameters.

Complete Chapter List

Search this Book: