Test Case Selection Using Feature Extraction and Clustering

Introduction

An efficient way to conduct regression testing is to find a minimal subset of test cases (Sumit, Rajesh, & Dhavleesh, 2016) so that the minimized test suite can be executed even with resource constraints (Mohamad, Joanna, & Philip, 2016), involving both time and cost. A suitable subset could be found during test case generation or after creating the test suite. Test suite size reduction results in quick conduct of testing. The process of finding a minimal subset is called test suite reduction, test suite minimization or test case selection, which results in a so called representative set and normally coverage based techniques are used to arrive at it.

Test suite contains test cases meant for varied program requirements. Out of evolution the test suite grows and few test cases may become redundant. Thus, not all test cases need to be used for conducting regression testing. Coverage based test suite reduction techniques help in keeping test cases based on the coverage requirement used and the redundant ones based on the coverage requirement are removed. But the test case removed because it is redundant with respect to a coverage requirement may not be redundant with respect to another coverage requirement. Further the coverage based test suite reduction technique does not address the dynamic behavior of software. There is no room to bring in the importance of the frequent usage of functionality, complex conditional block capable of revealing more faults and sequence of unusual branches taken, which shows the possibility of having fault. The execution pattern of software throws light on information like, what part of the software gets executed often, what are the functionalities used repeatedly, critical functions, coverage information, and detail about whether a branch is taken or not. Rather than reducing test suite based on coverage information (Gregory, Matt, Michael, & Mats, 2015), if the execution behavior of the software can be profiled and analyzed, then more relevant test cases which are more fault revealing can be selected. Identifying and using vital information for deciding or selecting relevant test cases plays an important role (Johny, Mor, Pnina, & Yaron, 2013).

Execution profile of a software contain the details of functionalities, blocks, branches, statements, conditions and the complex sequences of them exercised in the executions. The profile information collected can be analyzed using clustering technique (Michail & Nikolaus, 2013). Since clustering is a technique which helps in grouping candidates with similar behavior, test cases can be clustered based on execution pattern which are captured from execution profiles. Hence test cases can be selected based on dynamic behavior of software, rather than by coverage alone, which brings in hope for selecting more fault revealing test cases. This kind of selection is made possible by exploring the execution profile and identifying the complex combination of executions and rare sequence of executions which are more fault revealing. Execution profile based clustering technique based on function call profile has been used by Zhang et al. (2010).

The persisting necessity to conduct effective regression testing is the main motivation behind this work. The sole focus is to improve existing test case selection approach so as to assist in regression testing. The review on existing works highlighted the importance of finding and using new decision making parameters. To incorporate all possible aspects, multiple parameters need to be identified and used (Arunasalam & Vidyasagar, 2017). Based on that, in this work, an execution profile based clustering technique, based on three new features namely, function frequency, branches taken and block percentage of the test suite is used. The test cases are executed and these features are extracted from their execution profile. The test cases are then clustered using K-means algorithm based on these extracted features. From the resulting test case clusters, required number of test cases is selected using the selection heuristic. First, one test case from each cluster is selected, and if there are more unsatisfied requirements, more test cases are selected from the clusters. Cluster holding fault revealing test case and cluster having unusual number of test cases are taken into consideration since they exhibit that behavior, those test cases are selected.